The Question

When a psychology study reports that participants judged X as morally worse than Y, how many participants actually made that judgment? The answer, McManus and colleagues (2023) argue, is often far fewer than readers assume, and the gap between group-level findings and individual-level reality is one of the most consequential and least acknowledged problems in experimental psychology.

The Problem

Group-level statistical analyses test whether average responses differ between conditions. A significant result means the average in one condition is higher than the average in another. It does not mean that most individuals showed the same pattern. And in many cases, the proportion of individuals whose responses match the group-level finding is surprisingly small, sometimes constituting only a minority of participants (McManus et al., 2023).

This is the group-to-person generalizability problem. It arises because group statistics and individual-level patterns can diverge substantially, particularly when distributions are skewed, when there is large individual variability, or when the effect is driven by a subset of participants. The authors demonstrate that claims derived from group-level analyses can emerge even when not a single participant’s responses match the claimed pattern (McManus et al., 2023).

What the Research Shows

A survey of researchers and laypeople found that most interpret claims based on group-level effects as intended to represent most participants in a study, and most believe this ought to be the case when a claim is used to support a general psychological theory (McManus et al., 2023). In other words, both producers and consumers of psychological research assume that group findings speak to individuals. The authors document systematic cases in the experimental psychology literature where this assumption fails, where claims describe the responses of only a minority of participants.

Four experiments ruled out several methodological noise explanations, confirming that the problem is substantive rather than artifactual (McManus et al., 2023).

The Connection to Wider Debates

This finding connects directly to Uher and colleagues’ (2025) paper on psychology’s questionable research fundamentals reviewed earlier in this series, specifically to the ergodic fallacy: the assumption that patterns observed at the group level generalise to individuals. McManus and colleagues provide concrete, experimental documentation of how often and how badly this assumption fails in practice (McManus et al., 2023).

Why It Matters

Psychology’s stated purpose is to understand human psychological functioning. Most of its methods study groups and report averages. When group findings do not reliably describe individuals, the field’s findings are less about persons than they appear, and the theories built on those findings inherit the same limitation. The authors propose practical solutions for researchers to investigate and report person-level generalizability alongside group-level results (McManus et al., 2023).

Reference

McManus, R. M., Young, L., & Sweetman, J. (2023). Psychology is a property of persons, not averages or distributions: Confronting the group-to-person generalizability problem in experimental psychology. Advances in Methods and Practices in Psychological Science, 6(3), 1–23. https://doi.org/10.1177/25152459231186615