There must surely be a strong moral, ethical and indeed economic case that the core purpose of mental health services should be to support people in their recovery from mental distress and its myriad consequences. While recovery is now widely characterised as an individual and deeply personal process there are things which mental health services do which have the potential to either help or hinder people on that journey. While the focus on recovery in policy and practice has grown massively over the last twenty or so years, in the UK and worldwide, the evidence base to support recovery focused interventions is limited (a consistent theme in my previous Mental Elf blogs on this topic).

Recovery focused practice in mental health services is generally agreed as comprising a number of components including:

• a focus on people’s strengths,
• emphasising ‘personal’ recovery rather than solely focusing on reducing symptoms, and
• encouraging people to play a lead role in their own recovery.

When policy makers and planners want practitioners to do things in a certain way or to refocus their practices then training is often the tool of choice. Problematically, the evidence for the usefulness of staff training in realising improved outcomes for recipients of their services is not great. The ‘train and hope’ approach, a term coined by Stokes and Baer in 1977 to describe sharing information and skills in a highly structured environment in the hope that they will be generalisable to, and implemented in, everyday practices, has been shown to be particularly ineffective (Fixsen et al, 2005).

One high profile attempt to train new recovery focused skills within a workforce was the REFOCUS study which took place across two health trusts in England (Slade et al, 2015), one of a number of trials which have failed to demonstrate improved recovery outcomes. Given all this, it would be understandable if researchers chose to give recovery training interventions a wide berth. Undaunted Meadows and colleagues (2019) set out to test an adapted version of the REFCOUS training intervention in Australian community mental health services.

Methods

A total of 18 public mental health and community support services in Victoria, Australia were included. These services were further grouped into 14 clusters. Teams were paired within these clusters on the basis of their functions and characteristics. Each pair was then randomly allocated to receive the training in either the first or second year of delivery. This approach to cluster randomised controlled trials is known as a ‘stepped wedge’ approach. It means that recruitment and staff training took place in waves and that ultimately all teams received the training.

The intervention consisted of a manual, which was updated from the one used in the REFOCUS trial, an associated two-day training programme and follow up ‘active learning’ sessions offered on a monthly basis to training recipients. The main outcome was personal recovery measured using the QPR (Neil et al, 2009) for all people using services recruited to the study. A range of secondary outcomes, including wellbeing, perceived coercion and functioning, were measured in the second wave of recruitment only. Measures were completed at baseline and at follow up, which was at least 9 months after training delivery.

People in receipt of services were eligible for the study if they were receiving care and support within a team participating in the study.

Results

The PULSAR intervention was delivered to 190 staff across services. There was a high level of staff turnover in both the public mental health (68%) and community support services (50%). Two public mental health teams did not take up the post training ‘active learning’ opportunities, but for the teams who did the mean number of sessions was 8.1. In community support services these sessions were integrated within team meetings and therefore harder to quantify.

Recruitment targets of 18 service users per cluster were achieved with a total 942 people in receipt of services from participating teams recruited to the study at three time points. There was a good gender balance, half of the sample being between 30 and 49 years and most spoke English as a first language (87%). A quarter of participants were born outside of Australia.

The mean QPR score was 53·6 (SD 16·3) in the control group and 54·4 (16·2) in the intervention group (adjusted difference 3∙7, 95% CI 0·5 to 6·8; p=0·023). While this was a statistically significant improvement in recovery for people using services in teams who had received the intervention, only a small effect size was observed (d=0.23). Statistically significant differences were also observed when comparing recovery scores for people using services before and after staff training in both public and community service groups, but effect sizes were again modest. None of the secondary outcomes showed statistically significant improvement between the intervention and control groups but there were positive signals of improvement in nine out of ten measures.

Conclusions

The REFOCUS-PULSAR recovery training intervention shows considerable promise as one means to improve personal recovery outcomes for people in receipt of secondary care mental health services.

Strengths and limitations

Community mental health services are fluid and often highly pressured environments. Diminishing resources, increasing demands and shifting policy requirement can lead to high levels of staff turnover and uncertainty. This research team should be commended both for applying a rigorous and pragmatic research design in such a complex environment and also for having the faith to undertake research to genuinely test the effectiveness of a training-based intervention in trial conditions.

By adopting innovative methods and providing an intervention that was clearly valued by staff, the team were able to meet recruitment targets. They also showed that the intervention had an effect on personal recovery for people in receipt of services who had been exposed to the intervention. As the authors highlight, it’s rare for research into training based interventions to find any impact on patients, and in combination with wider challenges in identifying effects for recovery based interventions, this study offers renewed hope for advocates of recovery approaches and proponents of mental health staff training alike.

There is an argument that trying to train recovery values and practices within a staff team who are firmly embedded in existing service cultures and ways of working may be a poor investment in comparison to ensuring pre-registration training for new staff is designed around recovery principles, but these findings suggest both are important. Others have suggested that more disruptive approaches are more likely to lead to changes in practice towards a recovery focus than training alone. However, the PULSAR-REFOCUS intervention may have worked to achieve better outcomes for service recipients because it was indeed about much more than ‘teach and hope’ with practitioners left with new tools to implement through a detailed manual including an interview guide and care planning tools.

The authors are to be commended for their thoroughness and honesty in reporting their findings. Detailed appendices provide additional information on adverse events procedures, training quality assurance and adaptations as well as additional analyses. The authors are also honest in acknowledging that had their study adopted the same parallel group design as that used in the REFOCUS study that they would not have found an effect for recovery outcomes, something they may well have chosen to omit from their reporting. This type of transparent and open reporting of research findings makes replication of studies more possible and also builds much needed trust in the research community.

Direct comparison between PULSAR and the preceding REFOCUS study is also complicated by the studies employing different eligibility criteria for people in receipt of services. In the UK REFOCUS trial, eligibility was limited to people with experiences of schizophrenia, schizoaffective disorder or bipolar disorder, while this Australian PULSAR trial was open to anyone receiving services from participating teams. This would have generated a more transdiagnostic sample. Other service differences, like more rapid movement of people in and out of secondary services in Australia, should be borne in mind when making comparisons.

My one question about this this study was in relation to baseline assessments, with data gathered for up to 3 months after teams completed training. While this presumably maximised data collection, and was justified in the paper with reference to the Kirkpatrick four stage training evaluation model (Kirkpatrick & Kirkpatrick 2006), it seems like it may have biased the results in that service users may have been exposed to elements of the intervention prior to their baseline assessment (albeit any biasing would though have been towards a negative impact of recovery outcomes). Additionally, readers may also wonder about the long term impact of any training based intervention. Did everything regress back to standard practice after the researchers had packed up shop or was there a continued positive effect of recovery focused practices and outcomes? To answer that question would require further research.

Finding evidence for the effectiveness of training at level four in Kirkpatrick’s training model, i.e. does it improve outcomes for end recipients, is hard to achieve, not least because Randomised Controlled Trials are costly and complex to implement. However, it is only through this type of well controlled research that we can genuinely shift from hoping that training interventions have an effect, to providing robust and much needed evidence of improved experiences for service recipients.

Implications for practice

• This study suggests that it is possible to train large numbers of community mental health staff in recovery focused practices and that this training can benefit the personal recovery of people receiving services
• While the effect size was small, this is not unusual in this type of complex trial, particularly given the ever changing nature of the setting
• These findings offer new hope to proponents of recovery-based approaches and much needed evidence to support the wider dissemination of such approaches in adult services, in line with a pre-existing policy focus
• Future research should consider the long-term impact of such training based interventions on recovery outcomes.

Conflicts of interest

Simon Bradstreet is the former Director of the Scottish Recovery Network.

Refocus on Recovery #RonR19

The Refocus on Recovery 2019 conference aims to advance the field of recovery research and to create an international community of influence. This year in Nottingham on 3-5 September, the conference brings together leading researchers and people who use mental health services, their carers and informal supporters, mental health workers and professionals, and policy-makers and other stake-holders from across mental health services and wider systems.

Links

Primary paper

Meadows, G., Brophy, L., Shawyer, F., Enticott, J.C., Fossey, E., Thornton, C.D., Weller, P.J., Wilson-Evered, E., Edan, V. & Slade, M., 2019. REFOCUS-PULSAR recovery-oriented practice training in specialist mental health care: a stepped-wedge cluster randomised controlled trial. The Lancet Psychiatry, 0366(18), 103–114.

Other references

Fixsen, D. L., Naoom, S. F., Blase, K. A., Friedman, R. M. & Wallace, F. (2005). Implementation Research: A Synthesis of the Literature. Tampa, FL: University of South Florida, Louis de la Parte Florida Mental Health Institute, The National Implementation Research Network (FMHI Publication #231).

Kirkpatrick D.L. & Kirkpatrick J.D. (2006) Evaluating training programs. The four levels, 3rd edn. San Franciso, CA: Berrett-Koehler Publishers

Neil, S.T., Kilbride, M., Pitt, L., Nothard, S., Welford, M., Sellwood, W. & Morrison, A.P. (2009). The questionnaire about the process of recovery (QPR): A measurement tool developed in collaboration with service users. Psychosis, 1(2), 145–155.

Slade, M., Bird, V., Clarke, E., Le Boutillier, C., McCrone, P., Macpherson, R., … Leamy, M. (2015). Supporting recovery in patients with psychosis through care by community-based adult mental health teams (REFOCUS): a multisite, cluster, randomised, controlled trial. The Lancet Psychiatry, 0366(15), 1–12.

Stokes, T. F., & Baer, D. M. (1977). An implicit technology of generalization. Journal of Applied Behavior Analysis, 10, 349-367.

Photo credits

• Photo by Guillaume Bourdages on Unsplash
• Photo by Neringa Šidlauskaitė on Unsplash