Treatment Interaction Effects

Treatment interaction effects occur when the effect of one experimental treatment depends on the level or presence of another treatment.

Key points

Statsig frames interaction effects as the main statistical concern in Parallel A/B Testing: simultaneous tests may influence each other, cancel each other out, or create a combined effect that is different from either test alone ^[src-029].
The article defines the statistical check as asking whether the effect of one treatment remains consistent across all levels of the other treatment ^[src-029].
The recommended implementation is to create an interaction variable by multiplying the dummy variables for the treatment levels, then include that interaction term in a regression model ^[src-029].
If the interaction term is not significant, analysts can usually analyze the tests separately as independent experiments ^[src-029].
If the interaction term is significant, analysts should examine the treatment combinations directly and interpret the combined cells rather than only the isolated main effects ^[src-029].
The article notes that interaction analysis can be done on the overlap period even when parallel tests do not start and end on exactly the same dates ^[src-029].

^[src-029] Allon Korem and Oryah Lancry-Dayan — “You can have it all: Parallel testing with A/B tests” (2025-06-24)