The Question

The evidence that mechanical combination of assessment data, using algorithms and statistical models, produces more valid predictions of human performance than holistic combination by human judges is not new. It has been documented for decades. Yet practitioners continue to rely predominantly on human judgment. Neumann and colleagues (2023) investigated why, through surveys and focus groups across several Western countries.

The Study

Two surveys involving 323 decision makers in psychological assessment and HR practice, plus two focus groups, examined how practitioners combine information, why they do or do not use mechanical combination, and what would be needed to increase its use (Neumann et al., 2023).

What They Found

Most participants reported using holistic combination, typically in teams. The most common reasons for not using mechanical combination were practical and attitudinal: algorithms were simply unavailable in their practice contexts; stakeholders did not accept their use; practitioners did not quantify information in ways compatible with algorithmic processing; they did not believe the research evidence on evidence-based decision making; and they believed that combining holistic and mechanical approaches produced the best outcomes, a belief the evidence does not support (Neumann et al., 2023).

The finding about stakeholder acceptance is particularly telling. Even when practitioners themselves might accept the evidence for mechanical combination, the organisations, clients, and candidates they work with often do not, creating institutional barriers that are difficult to overcome regardless of what the research shows.

Why This Matters

The gap between what the evidence supports and what practitioners do is one of the most persistent problems in applied psychology. The evidence for mechanical over holistic combination is among the most robust in the field, yet it has made limited practical headway. Neumann and colleagues (2023) provide the most detailed recent account of why, identifying specific barriers that training, tool development, and stakeholder communication would need to address to narrow the gap.

Reference

Neumann, M., Niessen, A. S. M., Hurks, P. P. M., & Meijer, R. R. (2023). Holistic and mechanical combination in psychological assessment: Why algorithms are underutilized and what is needed to increase their use. International Journal of Selection and Assessment. Advance online publication. https://doi.org/10.1111/ijsa.12416