Both estimands could be termed the effect of a criminal record on the probability of a callback among Black and White applicants, yet the two are quite different. A design targeting causal interaction (Pager 2003) would randomly assign units (applicant–application pairs) to a cell of the 2 × 2 table that combines all values of both treatments. A design targeting effect heterogeneity would take applications in the real-world distribution for each subgroup and estimate the outcome they would realize if they signaled or did not signal a criminal record. Both estimands are of substantive interest.
I often see researchers confuse the effects they can measure in experiments vs observational studies. This fig from doi.org/10.1177/0003... is my go-to when explaining this:
#stats #causaleffect #experiments #DAGs #research