The Question

Cognitive ability tests are widely used in high-volume graduate and professional recruitment. They are commonly justified by reference to their predictive validity for job performance. Woods and Patterson (2023) present a critical review of that justification, arguing that both the validity evidence and the fairness case for cognitive ability testing in these specific contexts are weaker than is commonly assumed.

Five Critical Observations

The authors organise their critique around five specific contentions. First, cognitive ability tests show consistent evidence of adverse impact: candidates from certain demographic groups score systematically lower on average, with consequences for their access to graduate and professional occupations (Woods & Patterson, 2023). This is well-established in the literature and not seriously disputed.

Second, cognitive ability testing tends to be positioned early in high-volume recruitment processes, functioning as a screening hurdle before more comprehensive assessment takes place. This placement maximises the adverse impact of any group differences, because candidates screened out at this stage have no opportunity to demonstrate their capabilities through other means (Woods & Patterson, 2023).

Third, and connecting directly to the Sackett and colleagues (2024) and Steel and Fariborzi (2024) papers reviewed elsewhere in this series, recent meta-analytic work has substantially revised the estimated validity of cognitive ability tests downward from the historically cited figures. The corrected validity of around .22 or lower that contemporary research suggests is considerably less impressive than the .51 figure that has long been used to justify these tests (Woods & Patterson, 2023).

Fourth, the historical primary validity studies on which much of the evidence base rests have significant weaknesses, including sample characteristics, criterion measures, and design features that limit generalisation to contemporary graduate selection contexts.

Fifth, the examination of differential validity, whether cognitive tests predict performance equally well for different demographic groups, has been conducted in ways that are conceptually flawed, making it difficult to draw sound conclusions about whether the tests are equally valid across the groups they most affect (Woods & Patterson, 2023).

Why It Matters

The combination of modest validity, substantial adverse impact, and early placement in selection processes raises genuine questions about whether cognitive ability testing in graduate recruitment can be justified on evidence grounds, and whether its continued use may be actively maintaining and exacerbating social inequality in access to higher occupations (Woods & Patterson, 2023). The authors discuss strategies ranging from modifying how cognitive tests are used to removing and replacing them, and argue that the field needs to engage more honestly with both the empirical and the ethical dimensions of this question.

Reference

Woods, S. A., & Patterson, F. (2023). A critical review of the use of cognitive ability testing for selection into graduate and higher professional occupations. Journal of Occupational and Organizational Psychology. Advance online publication. https://doi.org/10.1111/joop.12470