The crisis in mindfulness research: have we been asking the wrong questions?

A review of mindfulness research in New Scientist highlighted long standing scientific problems; is it time for a new approach?

The crisis in mindfulness research: have we been asking the wrong questions?

How does science understand meditation
xvideTheo.com

Writing in New Scientist on June 5th Jo Marchant summarised the state of mindfulness research and practice. The investigation added some much-needed balance to the overview of medicalised mindfulness. The article confirmed the enduring presence of uncertainties in theoretical understandings and systemic methodological weaknesses. A discussion of the potentially harmful effects of meditation was especially welcome; most experienced meditation teachers know that practices can lead to beneficial or detrimental outcomes in practitioners.

However, the absence of greater historical insights left us with a snapshot rather than an overview of the current state of our scientific knowledge. For example, scientists have been criticising meditation experiments since the 1970s, but the weaknesses identified over 40 years ago can still be seen in contemporary research. The scientific study of meditation can be traced back at least 80 years; the first decades were relatively free of scientific uncertainty. By identifying the beginning of hesitancy in meditation research, we can better understand the current crisis in the science of mindfulness. Since 1975, an estimated 7,000 scientific papers investigating meditation have been published. The vast majority of this work has focussed on mindfulness, so should we be worried that we still don’t have a reliable scientific definition of it?

The evidence suggests that we (meditation scientists) have been trying to establish mindfulness’s psychological and clinical potential ahead of a stable understanding of what it is. We know from several strategic reviews that multiple ways of understanding mindfulness exist in the scientific literature. While each mindfulness experiment can offer us some new insights, findings are rarely confirmed through replication? When taking the long view of meditation research, medicalised mindfulness manifests within visible patterns of scientific progress. In its origins, medicalised meditation reflects a confluence between positivist and belief based knowledge systems. The current theoretical uncertainty in mindfulness research can be traced back to this convergence. If mindfulness has been developed as a bridge between spiritual and scientific understandings, do we have adequate ways of making sense of meditation as a human experience? The lack of stable definitions and replication suggests there are still significant gaps in our knowledge. The most pressing unanswered questions remain the most important, what is medicalised mindfulness, and how can we understand it?

What does the replication crisis mean for the science of meditation and mindfulness?

The scientific study of meditation has been limited by a replication crisis and a mindfulness crisis. What does this mean and what is the way forward for contemplative science?

Replication, an important element of the scientific method

For at least the last 20 years psychological science has been facing a replication crisis.1 For those who don’t know, the replication crisis reflects a deep-seated problem in how psychology carries out scientific investigations. In essence, it means that many psychological studies from the past may not be as reliable as we thought they were. This uncertainty has implications for the way psychology is conducted, and it may accelerate the declining public confidence in science more generally.

The replication crisis is visible in social sciences and medicine, but not all disciplines have been affected to the same extent. Although social psychology is regarded as having the most significant replication problem, the phenomenon is present in other areas such as the science of meditation. For an experimental study to be scientifically reliable, it generally has to be repeated, repeated by other scientists in alternative locations. If the results are the same, or at least very similar on each of these occasions, the scientific findings are much more likely to be reliable. However, if scientific claims cannot be replicated, it raises questions about how they were initially established, and the extent to which they can be generalised across populations. So if one scientific study found that regular meditation reduced the effects of hay fever, we’d expect to see the same results in other studies carried out in the same way. If not it could mean that there was an unusual characteristic in the first study or some problem in the method. It is for these reasons meditation scientists, teachers and practitioners are reevaluating what they know about the health benefits of meditation.

A failure to replicate doesn’t necessarily prove that scientific findings in the original study were not reliable, but it raises questions over the extent to which the claims are robust. So any isolated evidence for the health and wellbeing benefits of meditation has to be seen as a pilot study, preliminary in nature. In most cases, without replication, we cannot assume that findings from any individual study could apply to the general population.

For those of us working with meditation, the replication crisis is compounded because we are also facing a ‘mindfulness crisis’. The mindfulness crisis describes systemic problems in meditation research that go back 50 years. At least half a dozen studies published since 2015 have identified and described the meditation and mindfulness research crisis. Its main characteristics are conflicting theoretical understandings of meditation and methodological limitations which include low levels of replication. Although many, perhaps most scientific studies of meditation have been impacted by problems linked to the replication and mindfulness crises. The scientific enthusiasm for meditation technologies since the 1970s has been so great that one-off unreplicated claims for the benefits of meditation have not always been critically evaluated by the scientific community. As Van Dam and colleagues have demonstrated, this has led to the ‘hyping’ of preliminary evidence as robust scientific findings.2

Measures are being taken to address the replication crisis within psychology more generally. These initiatives have had a limited effect so far, and their impact will have to be evaluated over the longer term. To overcome the problems being experienced in Contemplative Science, there are three issues that need to be considered by the scientific and practice communities. Firstly the development of a system where unreplicated, preliminary findings are not treated in the same way as robust, replicated work. Secondly, address the pressing need to understand and resolve the known theoretical and methodological limitations. And finally, to review the procession of the scientific understanding of meditation since the 1930s to make sense of the current crisis and diagnose its underlying causes.

Notes

1 Maxwell, S. E., Lau, M. Y., & Howard, G. S. (2015). Is psychology suffering from a replication crisis? What does “failure to replicate” really mean?. American Psychologist, 70(6), 487.

2 Van Dam, N. T., van Vugt, M. K., Vago, D. R., Schmalzl, L., Saron, C. D., Olendzki, A., … & Meyer, D. E. (2018). Mind the hype: A critical evaluation and prescriptive agenda for research on mindfulness and meditation. Perspectives on psychological science, 13(1), 36-61.