Large sample size does not address selection bias; volunteer samples may systematically differ from non-volunteers in ways that affect treatment response.

[UNDERMINE → premise #1] Volunteer participants in platform experiments self-select based on willingness to modify their social media behavior, which may correlate with lower susceptibility to algorithmic influence. The framing requires effects to generalize to the broader US adult social media user population, not just to experiment volunteers. Therefore, Large sample size does not address selection bias; volunteer samples may systematically differ from non-volunteers in ways that affect treatment response. (Warrant: Sample size increases precision but does not address systematic differences between volunteers and non-volunteers that threaten external validity.)

• Volunteer participants in platform experiments self-select based on willingness to modify their social media behavior, which may correlate with lower susceptibility to algorithmic influence.
• The framing requires effects to generalize to the broader US adult social media user population, not just to experiment volunteers.

These are challenges this argument’s reasoning pattern must still withstand. Answering them on Isonomia strengthens the argument.

• Is the literature really agreed that defects of kind K bias inferences in direction B, or is the bias direction itself contested?Open
• Does study S actually have defect D, or is the description of S inaccurate?Open
• Is the expected magnitude of the bias from D large enough to overturn S's reported effect, or is the effect robust to plausible bias corrections?Open
• Has S (or a follow-up study) performed a robustness check or sensitivity analysis that addresses defect D directly?Open
• Is this critique applied consistently — i.e., would it apply to studies on the other side of the debate that share the same defect kind K?Open
• Is H supported by independent studies that do not share defect D, such that S's defect does not undermine H itself?Open

Cited by 1 · 1 contest