What does the replication crisis mean for the science of meditation and mindfulness?

The scientific study of meditation has been limited by a replication crisis and a mindfulness crisis. What does this mean and what is the way forward for contemplative science?

Replication, an important element of the scientific method

For at least the last 20 years psychological science has been facing a replication crisis.1 For those who don’t know, the replication crisis reflects a deep-seated problem in how psychology carries out scientific investigations. In essence, it means that many psychological studies from the past may not be as reliable as we thought they were. This uncertainty has implications for the way psychology is conducted, and it may accelerate the declining public confidence in science more generally.

The replication crisis is visible in social sciences and medicine, but not all disciplines have been affected to the same extent. Although social psychology is regarded as having the most significant replication problem, the phenomenon is present in other areas such as the science of meditation. For an experimental study to be scientifically reliable, it generally has to be repeated, repeated by other scientists in alternative locations. If the results are the same, or at least very similar on each of these occasions, the scientific findings are much more likely to be reliable. However, if scientific claims cannot be replicated, it raises questions about how they were initially established, and the extent to which they can be generalised across populations. So if one scientific study found that regular meditation reduced the effects of hay fever, we’d expect to see the same results in other studies carried out in the same way. If not it could mean that there was an unusual characteristic in the first study or some problem in the method. It is for these reasons meditation scientists, teachers and practitioners are reevaluating what they know about the health benefits of meditation.

A failure to replicate doesn’t necessarily prove that scientific findings in the original study were not reliable, but it raises questions over the extent to which the claims are robust. So any isolated evidence for the health and wellbeing benefits of meditation has to be seen as a pilot study, preliminary in nature. In most cases, without replication, we cannot assume that findings from any individual study could apply to the general population.

For those of us working with meditation, the replication crisis is compounded because we are also facing a ‘mindfulness crisis’. The mindfulness crisis describes systemic problems in meditation research that go back 50 years. At least half a dozen studies published since 2015 have identified and described the meditation and mindfulness research crisis. Its main characteristics are conflicting theoretical understandings of meditation and methodological limitations which include low levels of replication. Although many, perhaps most scientific studies of meditation have been impacted by problems linked to the replication and mindfulness crises. The scientific enthusiasm for meditation technologies since the 1970s has been so great that one-off unreplicated claims for the benefits of meditation have not always been critically evaluated by the scientific community. As Van Dam and colleagues have demonstrated, this has led to the ‘hyping’ of preliminary evidence as robust scientific findings.2

Measures are being taken to address the replication crisis within psychology more generally. These initiatives have had a limited effect so far, and their impact will have to be evaluated over the longer term. To overcome the problems being experienced in Contemplative Science, there are three issues that need to be considered by the scientific and practice communities. Firstly the development of a system where unreplicated, preliminary findings are not treated in the same way as robust, replicated work. Secondly, address the pressing need to understand and resolve the known theoretical and methodological limitations. And finally, to review the procession of the scientific understanding of meditation since the 1930s to make sense of the current crisis and diagnose its underlying causes.

Notes

1 Maxwell, S. E., Lau, M. Y., & Howard, G. S. (2015). Is psychology suffering from a replication crisis? What does “failure to replicate” really mean?. American Psychologist, 70(6), 487.

2 Van Dam, N. T., van Vugt, M. K., Vago, D. R., Schmalzl, L., Saron, C. D., Olendzki, A., … & Meyer, D. E. (2018). Mind the hype: A critical evaluation and prescriptive agenda for research on mindfulness and meditation. Perspectives on psychological science, 13(1), 36-61.

Author: Stephen

Neuropsychologist researching what happens when a spiritual practice (meditation) is translated to a psychological intervention; what is lost and what is gained from the curative potential? A PhD candidate writing the scientific history mindfulness. Also researching how compassion and explicitly nondual meditation methods influence our physical and mental health. Stephen has decades of personal practice in spiritual and secular forms of meditation, he has also been trained in the Himalayan Science of Mind and Perception (Tsema). Alongside the teaching and research of nondual methods, Stephen trains his own brain every day with Dzogchen practices.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s