Scaling Allen & Tucker's 10-day individual-level effect to 5–15% population-level contribution assumes linear extrapolation, no decay over 12 years, and uniform exposure persistence that the framing identifies as unestablished.
Scaling from individual-level short-run effects to population-level long-run aggregates requires assumptions about exposure persistence, treatment heterogeneity, and network spillovers that the framing identifies as missing in experimental literature.
Allen & Tucker (2025) measure a 2-point shift on a 100-point scale within 10 days in a volunteer sample during a high-salience election period.
Short-term A/B tests exhibit novelty effects and adaptation effects; short-term null results do not predict long-term effects after user adaptation to algorithm changes.
Is the literature really agreed that defects of kind K bias inferences in direction B, or is the bias direction itself contested?
Does study S actually have defect D, or is the description of S inaccurate?
Is the expected magnitude of the bias from D large enough to overturn S's reported effect, or is the effect robust to plausible bias corrections?
Has S (or a follow-up study) performed a robustness check or sensitivity analysis that addresses defect D directly?
Is this critique applied consistently — i.e., would it apply to studies on the other side of the debate that share the same defect kind K?
Is H supported by independent studies that do not share defect D, such that S's defect does not undermine H itself?