A recent study published in PLoS ONE conducted a reanalysis of a meta-analysis on Positive Psychology Interventions (PPI’s). A meta-analysis is, in simple terms, a statistical means of combining data from a lot of studies, and is an analysis of analysis. The results of a meta-analysis are often more robust than single studies as they combine data from multiple sources. While meta-analysis is undoubtedly no substitute for well-conducted research, the results of meta-analysis to provide an understanding of the broad underlying trend between variables.

The following blog, however, is not a review of meta-analytical techniques. Instead, it is to celebrate the growing trend in psychology to develop as a science, by replicating results and providing nuisance to findings. The replication crisis has meant that there is a shift in thinking within the psychological sciences, and we are now more interested in replicable results rather than one-off click bait findings. The article in Plos One encapsulates much of what is good about where the discipline is heading, and I laud the study for this reason.

For starters, PLoS ONE is open source. OPRA have long been advocates of free science, and the fact that PLoS ONE’s reputation is growing is a good thing. Secondly, the study represents an analysis of previously analysed data. People are now able to explore the data sets themselves and establish the underlying trend. One study, nor one review, is enough to draw robust conclusions, and the reanalysis of data represents a step forward in the discipline.

From a methodological perspective, the study addresses many of the concerns discussed in the discipline for a long time. Publication bias and small sample bias are two effects considered central to the replication crisis, and in this meta-analysis, we can see the impact of such biases. Moreover, the researchers have gone back to the primary studies, rather than relying on the previous data sources. Too often in our discipline, faulty findings are reshared, and the untruth becomes a psychological law.

Most importantly, the effects for the positive effects for the interventions are often subjective well being, while the impact for addressing more problematic psychological states such as depression are much smaller and non-significant. Drawing conclusions from these findings is difficult, as it may be the intervention is useful but not for a given population. Alternatively, there may be tweaks to the intervention that are required to make it more effective. The point is that science is, by its nature, iterative, systematic and long term. Psychology needs to make this shift, to become more scientific.  The current study is a step in the right direction, and I will leave the last words to the researchers:

“Accordingly, a comprehensive and transparent meta-analysis of all relevant studies of PPIs is necessary and is likely to have a major influence on the field. Such a meta-analysis is likely to allow for meaningful moderator analyses in answering questions such as: Is group administration more effective than individual administration? Are longer interventions more effective than shorter interventions? Are some types of interventions more effective than other types of interventions? Importantly, a comprehensive meta-analysis is likely to provide a more definitive determination of how effective PPIs are at increasing well-being.”

Reference

White CA, Uttl B, Holder MD (2019) Meta-analyses of positive psychology interventions: The effects are much smaller than previously reported. PLoS ONE 14(5): e0216588. https://doi.org/10.1371/journal.pone.0216588