Earlier this year Plos One took the step of retracting a well known and widely cited mindfulness related study.
This particular study is a first for the Science of Meditation blog. Whilst we have featured a number of papers that have highlighted methodological problems in meditation and mindfulness research, this is the first time that we have drawn attention to a retraction of a peer-reviewed study.
The basis for the retraction is outlined in detail on the Plos One website, but we have paraphrased the three main points.
The handling Academic Editor shared an affiliation with three of the authors, although this didn’t emerge until post-publication.
Two of the authors hold or had held positions at an institute offering mindfulness related products and services in clinical contexts.
The paper has a number of errors including pooling of results which led to double counting and incorrect effect estimates in figures contained in the study.
There’s not a lot more we need to add to the identified issues, they speak for themselves. However, when considered as part of the ongoing crisis in mindfulness research they make troubling reading.
A general defence used in cognitive psychology when the findings of mindfulness studies are criticised is, the peer review system is self-regulating. That when studies are found to be below the expected standard, they are usually rejected during review. Or at the very least other experts working in the field have the opportunity to raise concerns in print. This retraction challenges this basic notion. Significant issues with both the methodology and the editorial process can endure, thus, have the ability to influence the scientific and popular understanding of mindfulness. According to Google Scholar, this Gotink et al. study has been cited over 400 times, the citing publications, in turn, used by thousands more papers. The details provided on the Plos One website indicate the study has received 50,000 views.
Rather than simply criticize this study or the journal, I would like to ask what this retraction show us about the way that meditation technologies are being treated by clinical and scientific institutions?
Authors: Gotink, R. A., Chu, P., Busschbach, J. J., Benson, H., Fricchione, G. L., & Hunink, M. M.
Title: Standardised mindfulness-based interventions in healthcare: an overview of systematic reviews and meta-analyses of RCTs
Summary: This is a research paper that (at the time of writing) had been retracted by PLOS ONE.
“In light of the methodological issue and concerns about the validity of the study’s results, the PLOS ONE Editors retract this article. We regret that these issues were not fully addressed prior to the article’s publication.”
An extensive explanation of the reasons behind the retraction are published on the Plos One website which can be reached by following the link below.
New research suggests that compassion and empathy-based meditation and mindfulness may be able to increase prosocial emotions and behaviour.
Authors: Luberto, C. M., Shinday, N., Song, R., Philpotts, L. L., Park, E. R., Fricchione, G. L., & Yeh, G. Y.
Title: A Systematic Review and Meta-analysis of the Effects of Meditation on Empathy, Compassion and Prosocial Behaviors
Summary: Contemplative scientists may be aware of several recent meta-studies that have challenged the methodology and theoretical frameworks of meditation and mindfulness research. However a review by Luberto et al. recently found that compassionate and loving-kindness practices can positively influence prosocial emotions and behaviours. The key finding was that in the 26 reviewed studies, meditation was linked to a positive effect on self-reported and observed prosocial measures. Although this meta-study bucks the recent trend it should be noted that many of the known problems highlighted in earlier strategic reviews (methodological flaws, reliance on waiting list or no treatment control groups, weak theoretical frameworks) have not been fully addressed.
There is a growing understanding that repeated novel behaviours such as meditation training in meditation naive participants are correlated to new or adapted neural function and structure. This then places a much greater emphasis on using a meaningful control intervention from which to evaluate the effects of meditation training. The idea that compassion or empathy based meditation can increase prosocial feelings or behaviours when compared to ‘no intervention’ reflects an underlying weakness in meditation research generally. Comparing a meditation intervention to ‘no intervention’ offers us limited insight into the potential clinical value of a meditation technology. This synthesis included 15 waiting list or no intervention control group studies among the 26 featured experiments. But it should be stressed that significant results were found in studies using both passive and active control group interventions.
The paper included a Risk of Bias Assessment, a welcome inclusion given recent findings of the failure of scientific objectivity in some meditation and mindfulness research. Luberto and colleagues established the risk of bias using the Cochrane Collaboration guidelines. They reported that 11 of the 26 studies had a low risk of bias, 12 offered a medium risk with just 3 demonstrating a high risk. Eight potential domains for bias were evaluated for each of the reviewed studies, where the risk of bias in any of the domains could not be established a rating of ‘unclear’ for that domain was recorded. It should be noted that every study had a rating of ‘high’ or ‘unclear’ in at least one domain and the mean number of ‘unclear’ ratings was 40% of the total possible. Further, that in the risk evaluation of the 8 domains for each of the 26 studies (208 potential ratings in total), none were regarded as offering a medium risk of bias. Although I am unfamiliar with this approach to evaluating bias it would appear that an absence of data indicating potential bias in any domain is discounted from the overall classification. So individual studies with unclear data regarding potential for bias in areas such as blinding, incomplete results or selective reporting could still receive a low risk of bias rating!
“Is the absence of clear data masking real world risks of bias in meditation research?”
Stephen Gene Morris
Following a traditional approach, 26 papers with a total of 1714 participants were identified from academic and scientific databases. A selection criterion was used to deliver randomized controlled studies in a range of populations who were trained in loving-kindness or compassion meditation. Results for self-reported and observable outcomes indicated significant small to medium effects. Of note is that “subgroup analyses also supported small to medium effects of meditation even when compared to active control groups”. The study also contained insights into potential physiological and neural mechanisms linked to the meditation training. Limitations of this review included the wide range of meditation methods encompassed and the variable lengths, intensities and modes of training undertaken by the respective participants.
Within this paper (and much of the available research) definitions of compassion appear to be fluid. Put simply there are few signs that contemplative science draws upon authoritative definitions of compassion either in the meditation methods used or in establishing the effect of the training. There also appears to be potential for a disconnection between the concepts integrated into the meditation methods and the instruments designed to measure compassion and empathy. Whilst the data presented offers a cautiously optimistic picture of the potential of meditation to improve positive prosocial emotions and behaviours, established concerns over methodology and theoretical frameworks remain unresolved.
Strategic reviews are challenging the popular perception of the beneficial effects of mindfulness
How far can we trust meditation research?
No matter how I tried to write the headline it came out as provocative. My intention wasn’t to be controversial, rather I wanted to articulate concerns that have been rumbling around the science of meditation and mindfulness for decades. At the heart of this story are two important yet unresolved issues. Firstly how does psychology and neuroscience understand meditation and what do the results of meditation research really mean?
The limited prosocial effects of meditation is a recent systematic review of research undertaken by Ute Kreplin, Miguel Farias and Inti Brazil. The study has been discussed in the meditation community at some length so I’m not going to review it here. But to summarize, the positive effects of meditation on prosocial behaviours (compassion, empathy, aggression, connectedness and prejudice) in healthy adults were only observed in compassion and empathy scores. However increases to compassion were just seen when the meditation teacher was one of the co-authors of the research paper or when the study used a ‘passive’ control group (this means the control group were on a waiting list). These findings are suggestive of flaws and possibly ‘bias’ in some of those studies that demonstrated significant results. In an interview with Ute Kreplin published in the international Buddhist journal Tricycle, a number of broader issues have been highlighted, it’s those that I’d like to push around a little now. Leaving to one side the methodological flaws which are the main focus of the Tricycle interview, let me draw attention to the potential causes of the ongoing limitations in our attempts to evidence the effects of meditation.
It should be stressed that the Kreplin, Farias and Brazil paper is one of a number of reviews that came to similar conclusions, that many (possibly most) of the published studies reporting significant effects in non clinical populations had methodological and/or theoretical flaws. And as Kreplin hinted, published research tends only to be the tip of the iceberg, studies that fail to show measurable changes in meditators rarely see the light of day. So the examples analyzed in strategic reviews are not the full picture of meditation research, they offer a very selective (positive) account of the scientific landscape. And yet the common perception grows that meditation is a panacea able to deliver a range of desirable outcomes to almost anyone willing to practice a method.
“At this moment in time the science generally isn’t helping us to understand the benefits of meditation…”
Stephen Gene Morris
By way of transparency I should make it clear that I’m an experienced meditator and confident of the great benefits of the practice. My interest in contemplative science comes from the perspectives of both a trained cognitive psychologist and a practicing Buddhist. From my experience of teaching traditional meditation systems, it is unrealistic to claim that a few weeks of meditation practice automatically leads to ‘significant’ change. Some practitioners do progress rapidly, embracing the transformative potential of meditation, but others fall away after only a few weeks, sometimes disillusioned and unfulfilled. This is a difficult subject to address coming from a traditional meditation perspective, because judging or criticizing the progress of another practitioner is something of a taboo. But to enhance the wider understanding of meditation this point needs to be stressed. There is no reason to assume that the meditation method alone leads to change, the method is an integral part of a firmly established theoretical framework. The effects of meditation tend to be meditated by several factors such as, individual capacity, participant motivation and qualitative differences between the teacher or teaching systems.
The contemporary scientific investigation of meditation typically takes the reductionist approach, stripping out components that might confound the results of an experiment, such as variability in the method or differences in the environment. But isolating the cause (meditation method) and the effect (empirical change in the participant) is difficult, and in complex aspects of human behaviour such as empathy or compassion it may be beyond the scope of many experiments. Consider that large numbers of the participants in meditation studies are likely to be undergraduates ‘pressed’ into research projects, obliged to participate in return for course credits. If meditation doesn’t always work for the people who choose to attend classes in the wider community why should things be any different in an experimental setting?
The ‘expectation’ that a meditation method in itself leads to change is not supported by the human history. This idea may eventually be confirmed by science but the data gathered so far is inconclusive. We know that a number of meditation scientists are committed practitioners, so perhaps they have first hand experience of the benefits of meditation or mindfulness. Is this as Kreplin suggests, part of the problem? Could the experiential knowledge of the results of meditating introduce subconscious bias into research methodology? I’m a meditator I know about the benefits of regular practice but I can see dangers to the credibility of meditation systems if claims based on poor science are over-hyped. The lack of long term studies for secular forms of meditation should also be a serious concern.
The failure to establish robust findings in meditation research begs a further question, without reliable replicated science how does the delivery of meditation technologies continue to grow in society? If scientists are raising questions about the claims made in individual studies why isn’t this filtering down more into health care, public policy and the media? If meditation and mindfulness interventions cannot be shown to work, or deliver predictable results, confidence in meditation generally may decline. It might also lead to an erosion in the status of experimental psychology as a provider of independent and reliable data.
These few paragraphs are simply an introduction to the subject, the start of a very long road. It can be argued that the contemporary western scientific investigation of meditation began in the 1970s, since when perhaps as many as 10,000 studies have been published. But based on the findings from recent strategic reviews our scientific understanding of meditation is at a surprisingly preliminary stage.
The Kreplin, Farias and Brazil study can be found here.
The first episode of NDA offers an introduction into duality and non duality. It’s nice, easy to follow and uses everyday examples that makes the subject very approachable.
The first episode of NDA offers an introduction into duality and non duality. It’s informative, easy to follow and uses everyday examples that makes the subject very approachable. To sign up to the podcast visit the feed here.
Does mind wandering inevitably lead to unhappiness?
Authors: Killingsworth, M. A., & Gilbert, D. T.
Title: A wandering mind is an unhappy mind.
Summary: I’ve taken a step back with this study (chronologically speaking) because it predates the Jazaieri et al. paper that I recently reviewed. According to Google Scholar, this investigation has been cited by other researchers over 1500 times. If you read any scientific study linked to mind wandering since 2010 you can expect to find Killingsworth & Gilbert referenced. Three reasons why this work is so popular, firstly it was a big study, recruiting 2250 participants. Secondly, it took an innovative approach to the use of technology, using an iPhone app to track and record mind wandering in ‘real world’ scenarios. Finally, the study made some very strong statements which with hindsight, have perhaps oversimplified mind wandering. In particular, the idea that “a human mind is a wandering mind, and a wandering mind is an unhappy mind”.
The full purpose of mind wandering is not yet understood, its activity is largely observed in the Default Mode Network (DMN). The DMN is a collection of anatomically separate regions that are most active when we are not engaged in a specific externally focussed task, hence our ‘default state’. The research concludes that mind wandering is a cause of unhappiness and (staggeringly) even mind wandering to positive subjects doesn’t appear to improve self-reported happiness. Provided with this kind of context it’s no wonder that the idea of suppressive mind wandering approaches maintained popularity across experimental and clinical psychology. Killingsworth & Gilbert even supported their case by claiming that many religious and philosophical traditions link happiness to living in the here and now.
Many forms of meditation and mindfulness do work on training consciousness to rest in the present moment. But from the Tibetan Buddhist perspective to equate this with a conclusion that mind wandering is a cause of unhappiness is somewhat misleading. Traditional meditation is often set in a wider context, undertaken for the benefit of all beings (self and other). Any merit that is accumulated from resting in the present moment is typically dedicated to others, rooting the practice in the past present and future. The idea that a thought, rather than the reaction (attachment or aversion) to the thought is the cause of unhappiness isn’t supported by the theoretical frameworks of traditional meditation systems.
Abnormally high levels of mind wandering are likely to be clinically problematic, but even today we don’t know all the functions mind wandering might have. It’s considered to be linked to the narrative we make to understand ourselves in the wider world. There isn’t evidence that the process itself brings unhappiness, it may be that the quality of our self-generated narrative rather than mind-wandering per se might be the problem. For example, 1 in 4, 14-year-old girls in the UK demonstrate symptoms of depression, and that teenage depression is correlated with social media use. Can we say that mind wandering to social media content, rather than emotions linked to social media content is the root cause of any reported unhappiness?
Train in compassion to create a more positive outlook.
Authors: Jazaieri, H., Lee, I. A., McGonigal, K., Jinpa, T., Doty, J. R., Gross, J. J., & Goldin, P. R.
Title: A wandering mind is a less caring mind: Daily experience sampling during compassion meditation training.
Summary: The concept of mind wandering is well known to all of us. It’s the kind of drifting off that we experience when we are not concentrating on a specific activity or goal, people also call it daydreaming or spontaneous thought. If we have a challenging task that requires our full attention we tend to do little or no mind wandering, and conversely, when we are mind wandering we are much less able to concentrate on a task. The two neural networks responsible for task functions and mind wandering respectively are thought to be negatively correlated, when one is more active the other in more passive and vice versa. The benefits of better concentration and task performance are obvious, but we are starting to see that mind wandering may also have a key role to play in our health and wellbeing. This study is one of the few investigations into meditation that acknowledges that mind wandering may have a positive role to play in our lives.
This study investigated the effects of nine weeks of compassion training on 51 adults. As part of my own research into meditation and mind wandering, I have revisited the paper. There are two conclusions that I’d like to draw your attention to. Firstly that the meditation was linked to a decline in mind wandering to neutral topics but an increase towards pleasant topics. That meditation can lead to decreased mind wandering is well known, the highlight of this study is that meditation seemed to change the type of mind wandering. This is highly suggestive that mind wandering has both a qualitative and quantitative aspect. That some forms of mind wandering might actually be beneficial in some way, therefore suppressing mind wandering generally might not of itself be a useful target of any wellbeing intervention.
The evidence that compassionate meditation can naturally draw the mind away from the negative and towards the positive could have profound implications for our health. Mind wandering is spontaneous, it’s not consciously constructed if compassionate meditation leads to a natural increase in positive thoughts, it indicates an association with a range of other cognitive processes. This view is supported by a second finding from the research, that compassionate meditation is linked to augmentation in caring behaviours for oneself and others.
The default mode network has a crucial but poorly understood role in how meditation influences brain structure and function. This paper sets out some of the current thinking regarding self-generated thought.
Authors: Andrews‐Hanna, J. R., Smallwood, J., & Spreng, R. N.
Title: The default network and self‐generated thought: component processes, dynamic control, and clinical relevance
Summary: It is frequently suggested that neuroscience is still in its infancy, this becomes patently clear when you start to consider how little we know about the default mode network (DMN). The DMN, also known as the default network (DN) or the task-negative network (TNN) is most active when humans are in a resting state. In short, the DMN is the network that takes over when we are not actively engaged in a specific task. Surprisingly it was assumed that the brain was resting when not engaged in an externally focussed activity. This assumption was surprising because scientists know that their brains are capable of complex processes such as mind wandering when they are not reacting to the external environment. However, only when it was demonstrated that functional brain activity could reach similar levels in task and non-task modes did the investigation into the DMN begin in earnest. This has particular relevance for meditators and contemplative science, as the DMN is often the direct and indirect target for meditation methods.
Andrews‐Hanna, Smallwood and Spreng produced a review of the leading findings linked to the DMN, which they describe as an anatomically diffuse global network. Their primary focus is the DMN and self-generated thought, thought that arise without external sensory stimulus. Describing much of the recent research in the field they conclude that the DMN plays an integrated role in a wide range of neurological functions. Thus both normal and abnormal mental health is dependent on activity and functional connectivity within the DMN and links to other neural networks. The paper provides a useful background to contemplative scientists looking for an understanding of how meditation might influence human behaviour.