Virtual reality (VR) is pretty cool. I should know; I’ve had an Oculus Rift to play with for the past few days. In mental health (where, let’s be honest, there aren’t many ‘cool’ things) VR is becoming more and more popular as a way of delivering treatment. Previous Elf blogs have covered research looking at how effective it might be in the treatment of persecutory delusions, paranoia in those living with psychosis, and social anxiety disorder.
But no matter how cool something is, no matter how engaging it might seem, it can only be effective if people will keep using it. This is a typical challenge in digital mental health, where there is a need for a balancing act between how engaging, effective and efficient a product is.
This blog looks at a meta-analysis by Benbow and Anderson (2018) that used previous research to compare virtual reality exposure therapy (VRET) with real life (otherwise known as ‘in vivo’) exposure therapy (ET) for anxiety disorders. The authors were particularly interested in seeing whether there was a difference in how many people dropped out (also called ‘attrition’ or ‘non-adherence’) and their reasons.
Exposure therapy involves people facing their fears in a real world situation alongside a therapist, which can be extremely challenging and potentially distressing. It was hypothesised that fewer people would drop out of VRET as it is less intense.
Digital mental health interventions often struggle to be effective because people stop using them.
Methods
This study explored, through a meta-analysis, attrition (drop-outs) from studies using VRET for anxiety disorders. The authors compared this against ET in the real world (in-vivo) and explored reasons why people may or may not continue to use VRET.
Using the PRISMA guidelines for meta-analyses they used an electronic search using a combination of the following terms: virtual reality, VRE or VRET and anxiety, anx*, exposure, phobia, specific phobia, public speaking, panic disorder, panic, agoraphobia, generalised anxiety disorder, GAD, generalised anxiety, SAD, social anxiety disorder, social phobia, or social anxiety.
Studies were excluded if participants were under the age of 18, were case studies or only delivered a single treatment session, and if they did not have enough data. This led to the inclusion of 46 articles and 1,057 participants from an initial 1,688 articles. They excluded those that did not have a therapy comparison group when comparing dropout in VRET and ET, resulting in 13 studies and 756 participants.
The authors used a Mantel-Haenszel (MH) odds ratio to calculate the number of participants who dropped out compared to those who completed treatment for each group. They also assessed the quality of the articles, coding for selection bias (how participants were assigned), diagnostic methods (how they were identified) and treatment adherence (reviewing whether people continued treatment).
The meta-analysis itself was done using software and reasons for not dropping out (moderating variables) were explored to see whether they contributed to higher or lower drop-out rates. These included demographic variables like gender, anxiety disorder, virtual reality alone or with CBT, and anything offered between sessions, e.g. homework.
Virtual reality exposure therapy helps people come to terms with fears or phobias by gradually exposing them to increasing amounts of the stimulus over a set period of time, with support from health practitioners.
Results
- In their analysis the authors found that no one single study dominated the effect size
- However, when comparing VRET and in-vivo ET, the odds ratios ranged from 0.70 to 4.64 with a summary MH odds ratio of 1.09 (p=0.66, which is not significant)
- Dropout rates varied from 1.7% to 41.4%
- Those that provided interventions between sessions (e.g. homework) had significantly lower dropout rates when compared to those that discouraged them
- The number of sessions, inclusion of CBT and disorder being treated were all found to be unable to significantly explain the differences in dropouts
- Interestingly, the authors also looked more broadly at the reasons for dropping out reported within the studies. The most reported cause was being unable to immerse in the VR environment, followed by cybersickness, vision complications and discomfort interacting with the therapist when they were unseen.
This review suggests that virtual reality exposure and in vivo exposure therapy have similar rates of drop-out.
Conclusions
The authors conclude that VRET does not lead to lower drop-out than in-vivo exposure therapy, that an estimated 16% of participants drop out of VRET, and that this is better explained by the inclusion of interventions such as homework between sessions. Their analysis also suggested there are significant differences in reasons given for dropping out of VRET compared to ET.
Benbow and Anderson (2018) suggest that future research should explore how choice may impact drop-outs, as VRET may be more likely to be chosen over in-vivo exposure therapy (Opris et al., 2012). The authors also suggest trying to mitigate drop-outs early in treatment as number of sessions, and a review of the studies included, suggests that this is when people are more likely to leave. Although dropout rates were similar across VRET and in-vivo ET, the reasons were not, so interventions should mitigate non-engagement based on their choice of delivery. Lastly, the improvement of adherence to VRET linked to provision of between session interventions indicates that the working alliance between service user and therapist could be important and the authors call for further research into this.
The authors recognise the limitations of their study in exploring the impact of demographic characteristics on drop-out and the difficulty evaluating study quality.
Drop-out rates were similar across virtual reality and in vivo exposure therapy, but the reasons people dropped out were different, so interventions should mitigate non-engagement based on their choice of delivery.
Strengths and limitations
Considering that difficulty feeling immersed in the VR was the most commonly given reason for dropping out, I found it interesting that no mention was made of how old some of the studies were and that this was not factored into the meta-analysis. The oldest study included was from 1995 (Rothbaum et al., 1995) whilst the most recent was from 2017 (Bouchard et al., 2017). It is fair to say that the VR experience has come a long way in that time. Similarly, they did not consider the environment in which the VRET was delivered or the type of device it was delivered through. It would have been useful if Benbow and Anderson (2018) had highlighted that there were more reasons reported for dropouts from VRET when discussing their conclusions. It is also not clear whether the reasons for dropping out were self-reported by participants or reported by researchers.
Virtual reality techniques have changed dramatically in the last 20 years, so it’s likely that the evidence in this field will develop quite quickly over the coming years.
Implications for practice
One of the key questions arising from this meta-analysis is whether the reasons for dropping out of VRET versus those for in-vivo ET make one or the other a better option. Considering 54 out of 93 reasons given for dropping out of VRET were for reasons connected to the VR itself rather than the ET, this would indicate potential to increase the number of people treated successfully. However, there were still 39 who dropped out despite it being delivered using VR and the difference in attrition rates was negligible. This might suggest that cost and choice are better reasons to employ VRET, alongside the obvious importance of efficacy. A previous meta-analysis indicates it is as effective as real world ET (Opris et al., 2012), which suggests that further research should prioritise exploring whether it is preferred by service users and if so, why.
This meta-analysis seems to suggest that no single study, if removed from the analysis, changed the effect size. This essentially means that none of the studies had really low or high attrition that might sway the results. But does this mean that the VR itself doesn’t have a role to play? If a number of them used terrible VR (and remember some of the studies are over 20 years old) leading to higher dropouts, this statistical analysis would not have been sophisticated enough to identify that.
What I find most interesting, however, is how we can learn from the reasons for dropout from VR and in-vivo exposure therapy. It is unfortunate that so many studies do not report on this, especially considering how important it is for development of more engaging technologies. It would also have been interesting to know whether these reasons were attributed by researcher or by participants. The way the results were presented in this study also made it difficult to explore the reasons for drop-out across the comparative studies versus those only addressing VRET. This is important as 5 participants in the ET treatment arm dropped out because they wished to try VRET and 2 participants did the same when randomised to VRET. The role of preference and choice is considered by the authors to be important and yet it is not explored further. Rates of dropout attributed to VR versus ET may also have been a useful inclusion in their analysis, particularly in the studies that reported reasons for dropout.
Future research should find out whether mental health service users prefer virtual reality exposure therapy or in vivo exposure therapy.
Conflicts of interest
Aislinn is currently a Research Fellow at MindTech, University of Nottingham, exploring the implementation and sustainability of gameChange, an innovative virtual reality therapy for people with psychosis and winner of the first NIHR i4i Mental Health Challenge.
Links
Primary paper
Benbow AA, Anderson PL. (2018) A meta-analytic examination of attrition in virtual reality exposure therapy for anxiety disorders. Journal of Anxiety Disorders. http://doi.org/10.1016/j.janxdis.2018.06.006
Other references
Bouchard S, Dumoulin S, Robillard G, Guitard T, Klinger E, Forget H, … Roucaut FX. (2017) Virtual reality compared with in vivo exposure in the treatment of social anxiety disorder: A three-arm randomised controlled trial. British Journal of Psychiatry, 210(4), 276–283. http://doi.org/10.1192/bjp.bp.116.184234
Opris D, Pintea S, Garcia-Palacios A, Botella C, Szamosközi Ş, David D. (2012) Virtual reality exposure therapy in anxiety disorders: a quantitative meta-analysis. Depression and Anxiety, 29(2), 85–93. http://doi.org/10.1002/da.20910
Rothbaum BO, Hodges LF, Kooper R, Opdyke D, Williford JS, North M. (1995) Effectiveness of computer-generated (virtual reality) graded exposure in the treatment of acrophobia. American Journal of Psychiatry, 152(4), 626–628. http://doi.org/10.1176/ajp.152.4.626
Photo credits
- Photo by Hammer & Tusk on Unsplash
- Photo by Kai Pilger on Unsplash
- Photo by Hammer & Tusk on Unsplash
- Photo by Franck V. on Unsplash
- Photo by rawpixel on Unsplash
- Photo by Vladislav Skripchenko on Unsplash
