Treatment Interaction Effects

Treatment Interaction Effects

Treatment interaction effects occur when the effect of one experimental treatment depends on the level or presence of another treatment.

Key points

  • Statsig frames interaction effects as the main statistical concern in Parallel A/B Testing: simultaneous tests may influence each other, cancel each other out, or create a combined effect that is different from either test alone [src-029].
  • The article defines the statistical check as asking whether the effect of one treatment remains consistent across all levels of the other treatment [src-029].
  • The recommended implementation is to create an interaction variable by multiplying the dummy variables for the treatment levels, then include that interaction term in a regression model [src-029].
  • If the interaction term is not significant, analysts can usually analyze the tests separately as independent experiments [src-029].
  • If the interaction term is significant, analysts should examine the treatment combinations directly and interpret the combined cells rather than only the isolated main effects [src-029].
  • The article notes that interaction analysis can be done on the overlap period even when parallel tests do not start and end on exactly the same dates [src-029].

Related entities

Related concepts

Source references

  • [src-029] Allon Korem and Oryah Lancry-Dayan — “You can have it all: Parallel testing with A/B tests” (2025-06-24)

Robin Cartier perspective

This page is part of Robin Cartier's working AI knowledge graph: a practical research layer for production AI, recommendation systems, experimentation, GEO, and agentic web readiness.

The useful next step is to connect this concept back to applied product leadership and operating models.

Recommended next

Keep reading from this thread

From 491 indexed pages and articles.

  1. Wiki concept Experiment Statistical Power The ability of a test to detect a real effect when that effect exists. Key points Statsig's parallel-testing article Related by effect
  2. Wiki concept A/B Testing vs Bandits A/B testing versus bandits is an experimentation trade-off: fixed-allocation tests are better for clean inference and long-term decisions, while bandit tests are Related by ab testing
  3. Insight AI Measurement and Experimentation How to measure AI product impact with evals, adoption metrics, online experiments, guardrails, and cost tracking Readers have engaged with this next