Over the last few years, there have been a few posts in the woodland on studies that one way or another are about mental health recovery. They cover diverse topics including singing for recovery, social network mapping and the validation of a research measure.

I have previously contributed two blogs with a recovery flavour. Most recently, I wrote about an important qualitative study looking at the views of people on service reform and recovery over a twenty year period, concluding that if recovery is to flourish then it is vital we better hear the voices of people who appear to be benefitting the least from its values and practices. I also blogged about a useful review of the CHIME conceptual framework, which highlighted the potential for recovery research and development to focus too strongly on the positives of recovery, at the expense of the significant challenges people have in moving on with their lives.

Recovery is a subject I care very much about having spent a large part of my career raising awareness of recovery and encouraging the adoption of recovery-based approaches. I also have more than a passing interest in research and am firmly committed to the need to underpin improvement efforts with research evidence of the highest standard. Given these interests I was excited to come across a recent systematic review of longitudinal research on recovery (Thomas et al, 2017).

Systematic reviews are widely considered be at the top of the evidence pyramid (Murad, 2016), particularly where they include a meta-analysis of results across studies, as is the case in the Thomas review. By synthesising findings across included studies, systematic reviews generate new evidence which is greater than the sum of its parts. Such reviews should provide exactly the type of compelling evidence needed to underpin and justify efforts to increase a focus on recovery with decision makers and of course people personally affected by mental health problems.

Methods

The PubMed and PsycINFO databases were searched for papers using the following search terms:

‘longitudinal’
AND
(‘serious mental illness’ OR ‘schizophrenia’)
AND
(‘recovery’ OR ‘empowerment’ OR ‘hope’)

Papers in English were included where studies employed a quantitative measure of either ‘person oriented’ recovery (the authors describe how this is distinct from what they describe as ‘functional recovery’), hope or empowerment. These measures also had to be repeated at two or more time points to give an idea of change in recovery over time. The method of extracting data from selected studies and of assessing included studies for potential bias and study quality is described.

For the meta-analysis (a statistical analysis combining the results of multiple studies) a subset of articles identified in the main search were selected. These had to include a randomised controlled trial (RCT) of a recovery-oriented intervention. Data needed for an effect size calculation of this subset of studies was extracted along with additional information on other factors that might have been playing some role in determining outcomes. These included, for example, characteristics of the research participants and information about how interventions were delivered.

Results

Twenty three studies were included in the systematic review, seven of which were included in the meta-analysis. The majority of studies were based in the United States (n=15) with participants being predominantly middle aged, male and white. There was a broad range of sample sizes in the included studies. Twelve had sample sizes of over 100 with the largest study including 500 participants. Most of the recovery interventions took place weekly for anything between 8-83 weeks and all followed a set manual or curriculum. People leading the interventions were evenly split between ‘mental health professionals’ (35%), ‘peer specialists’ (35%) and a combination of both professionals and peers (29%).

A high proportion of included studies were rated as having a high risk of bias. For example, half of studies either did not include the random allocation of research participants or had no concealment of allocation.

Results of the meta-analysis of 7 included RCTs showed an average standardised mean change effect size of 0.24 for the recovery intervention. This means that there was a statistically significant but small effect on included outcomes across studies for people in the recovery intervention arm when compared to control groups. To put it in even simpler terms, there was evidence that the recovery interventions had an effect across the studies. This positive effect for recovery interventions was also maintained when measures were repeated at a follow up point after the intervention was completed.

Further analysis suggested that the only variable to have a statistically significant controlling effect on recovery outcomes was who led the intervention, with combined peer and professional delivery showing the greatest difference in recovery outcomes when compared to peer or professional delivery.

Conclusions

The authors conclude that we need more high quality research on recovery outcomes over time, which applies more consistent means of recovery measurement. They also conclude there is adequate evidence to support the wider dissemination of manualised recovery focused interventions in mental health services. Finally, they call for increased partnerships between mental health professionals and peer providers in the delivery of recovery interventions.

Strengths and limitations

As anyone who has ever attempted to undertake a systematic review will know, it is an activity not to be taken lightly, so the authors deserve credit for undertaking this broad and ambitious review. Importantly, they clearly describe and report their work to assess for quality and bias in included studies and find considerable grounds for recommending improvements in study quality. This is an important finding in its own right. This need for improved research is heightened, given the emphasis on recovery in policy and practice terms tends to run ahead of the supporting evidence. Systematic reviews can give an important state of the nation type overview and highlight shortcomings to be addressed. However, in my view the shortcomings of the current study render it a missed opportunity for recovery research.

When I think of the word longitudinal in relation to recovery, I think of the work of researchers like Courtenay Harding and the Vermont longitudinal study, which showed many more people with a schizophrenia diagnosis were recovering than had previously been thought, albeit over a long period (Harding & Brooks, 1987). I think of the WHO studies spanning decades which suggested better long-term outcomes in non-Western study settings (Harrison et al, 2001). I think of research that follows people over extended periods to review the progression of specified conditions. This review, however, is clearly concerned with the effectiveness of various recovery-focused interventions, which generally have short follow up periods after the completion of the intervention.

You may have noticed earlier that the search term longitudinal was followed by the term AND. This means it was a required search term and papers that did not include it would have been excluded. Unfortunately, ‘longitudinal’ is not necessarily a word that people leading the type of intervention research featured in this review would include in their papers. For example, the most high profile RCT of a recovery intervention in the UK to date was the REFOCUS trial (Slade et al. 2015). This paper, quite understandably, did not include the word ‘longitudinal’ in the title or abstract, and so would not have been found when the authors did their literature search, and therefore was excluded from the current review. Other seemingly relevant studies excluded from this review include research on Wellness Recovery Action Planning in Ireland (O’Keeffe et al. 2016) and Hong Kong (Mak et al. 2016). All three of these studies failed to show an effect for the main recovery outcomes, so would clearly have had a negative influence on the results of this meta-analysis if included.

Rerunning the search without the term longitudinal would I’m sure highlight more omissions of relevant papers, but doing so might be hard to achieve in practice. While the supplemental paper does include a flow diagram, that might more helpfully have been included in the main paper, there is no detailed description of the search strategy (as is advertised in the main paper). For example, were the search terms applied to article titles or titles and abstracts? This may all seem like bloggers pedantry, but a key characteristic of systematic reviews is that they should be replicable by other researchers and based on the information provided in the paper I was unable to replicate search results.

There are other problems with this review. For example, why include and aggregate three different outcomes? While hope and empowerment are widely agreed to be contributory towards the evolving construct of personal recovery (Leamy et al. 2011) they are certainly not equivalent. The majority of papers in the meta-analysis actually included all three variables so it’s hard to know what effect their aggregation would have had.

This all matters because, as mentioned earlier systematic reviews are generally seen as the ‘highest form’ of evidence and are much beloved by policy makers and people who decide how scant resources should be allocated. On the basis of this review, those same decision makers would be unlikely to generate much enthusiasm for recovery focused interventions given the artificially limited number of included studies and modest effect sizes.

Summary

• This systematic review of recovery interventions is described as being about longitudinal recovery outcomes when in fact it is largely concerned with recovery-focused interventions.
• Modest effects were found in favour of recovery interventions when compared to control groups. However, the selection of search terms for the review led to the omission of relevant papers, which means findings may not represent the best available evidence on recovery interventions.

Links

Primary paper

Thomas, E. C., Despeaux, K. E., Drapalski, A. L., & Bennett, M. (2017). Person-Oriented Recovery of Individuals With Serious Mental Illnesses: A Review and Meta-Analysis of Longitudinal Findings (PDF). Psychiatric Services, 69(3), 259-267

Other references

Harding, C. M., G. W. Brooks, et al. (1987). The Vermont longitudinal study of persons with severe mental illness: I. Methodology, study sample, and overall status 32 years later. American Journal of Psychiatry 144(6): 718-726.

Harrison, G., K. Hopper, et al. (2001). Recovery from psychotic illness: a 15- and 25-year international follow-up study. British Journal of Psychiatry. 178(6): 506-517

Leamy, M., Bird, V., Le Boutillier, C., Williams, J., & Slade, M. (2011). Conceptual framework for personal recovery in mental health: systematic review and narrative synthesis. The British Journal of Psychiatry : The Journal of Mental Science, 199(6), 445–52.

Mak, W. W. S., Chan, R. C. H., Pang, I. H. Y., Chung, N. Y. L., Yau, S. S. W., & Tang, J. P. S. (2016). Effectiveness of Wellness Recovery Action Planning (WRAP) for Chinese in Hong Kong. American Journal of Psychiatric Rehabilitation, 19(3), 235–251.

Murad, M., Asi, N., Alsawas, M. & Alahdab, F. (2016). EBM – New Evidence Pyramid. Evidence Based Medicine, 21(0), 125–126.

O’Keeffe, D., Hickey, D., Lane, A., McCormack, M., Lawlor, E., Kinsella, A., … Clarke, M. (2016). Mental illness Self-Management: A randomised controlled trial of the wellness recovery action planning intervention for inpatients and outpatients with psychiatric illness. Irish Journal of Psychological Medicine, 33(2), 81–92.

Slade, M., Bird, V., Clarke, E., Le Boutillier, C., McCrone, P., Macpherson, R., … Leamy, M. (2015). Supporting recovery in patients with psychosis through care by community-based adult mental health teams (REFOCUS): a multisite, cluster, randomised, controlled trial. The Lancet Psychiatry, 0366(15), 1–12.

Photo credits

• Photo by Alexander Andrews on Unsplash
• Photo by rawpixel on Unsplash
• hobvias sudoneighm CC BY 2.0
• Evidence pyramid was produced by HLWIKI Canada: http://hlwiki.slais.ubc.ca/index.php/File:EBMpyramid.gif