Individual-level randomization does not systematically underestimate population-level effects when treatment and control groups share network connections, as spillovers would contaminate the control group and bias results toward null.
The Guess et al. (2023) experiment included users embedded in real social networks where treated users' changed exposure could influence untreated network contacts.
If network spillovers were substantial, control group members would be partially treated through their connections to treated users, biasing the experiment toward finding null effects.
The null result therefore either reflects true absence of effect or underestimates the true null due to spillover contamination, not underestimation of a positive effect.
Is the literature really agreed that defects of kind K bias inferences in direction B, or is the bias direction itself contested?
Does study S actually have defect D, or is the description of S inaccurate?
Is the expected magnitude of the bias from D large enough to overturn S's reported effect, or is the effect robust to plausible bias corrections?
Has S (or a follow-up study) performed a robustness check or sensitivity analysis that addresses defect D directly?
Is this critique applied consistently — i.e., would it apply to studies on the other side of the debate that share the same defect kind K?
Is H supported by independent studies that do not share defect D, such that S's defect does not undermine H itself?