We’re not strangers to self management and peer support in the woodland. These topics have been subject to much discussion and debate, often centred on the apparent problem that, while many people value support from peers and taking control of their own recovery, hard clinical evidence of whether formalised approaches offer anything in addition to ‘treatment as usual’ is hard to come by.
In this blog we take an example of a peer-led self management training programme for people with severe mental illness and review three related papers that report on:
- The development and delivery of the programme
- An evaluation of its effect
- Its cost effectiveness
The peer-led self-management intervention
This project took place in Wales and was led by the Mental Health Foundation. The overall aim was to develop, deliver and evaluate a coherent, testable intervention for self-management for people with a severe psychiatric diagnosis. It built on a programme already developed and delivered by Bipolar UK.
In Crepaz-Keay and Cyhlarova 2012, the service user-led approach to developing the intervention is described. Twenty-four people who had used secondary mental health services took part in a 4 day residential workshop, facilitated by service user research and development experts. The workshop participants successfully argued for the inclusion of peer support in the self-management training course.
The model for the intervention that emerged from the workshop was a three stage intervention over nine months:
- Two days of training with a focus on goal setting and problem solving, including:
- An explanation of the principles of self-management
- The purpose and content of the training and peer support
- Exercises to support goal setting and problem solving
- Support to set personal goals
- Six half day follow up sessions came after, usually fortnightly, to review progress towards goals and collaborate on problem solving. The aim was for the facilitation level to reduce as the groups become more self-directed
- There was also on-going peer support, at least monthly for six months, which was self-led by the group, with only administrative and financial support offered by the programme, to encourage meetings to take place
Following this development process, the draft training manual and participant materials were reviewed by a separate service user group. The course was then piloted twice, with service user facilitators, and amended before the main delivery programme started, which incorporated the evaluation.
Over the course of around two years, 62 courses were delivered with 647 participants across Wales, and each time the courses were led by course participants from a previous course.
In the design workshop, service users recognised the essential importance of peer support in self-management.
Results
The evaluation of the intervention
The evaluation (Cyhlarova et al, 2015) was a pre- and post-test design assessing wellbeing and health promoting behaviour at baseline, six and 12 months in a single cohort. The measures used were the 14 item Warwick-Edinburgh Mental Wellbeing Scale (WEMWBS) and an adapted version of the Health Promoting Lifestyle Profile II (HPLP II).
240 volunteers completed the measures at baseline, with 55% (132) followed up at six months and 30% (72) followed up at 12 months.
Mean scores on the WEMWBS increased between baseline and six months and showed a further increase by 12 months. T-tests indicated these differences were significant.
Mean scores on the HPLP II increased between baseline and six months and a further increase by 12 months. Again, t-tests indicated these differences were significant.
Given these findings, the authors report that:
The long term improvements in self-reported wellbeing and health promoting lifestyle following the training suggests that this self-management course may be effective in changing people’s behaviour.
The authors conclude that the peer-led self-management course may be effective in changing people’s behaviour, but the lack of a control group or randomisation make this conclusion unfounded.
The economic evaluation of the intervention
The ‘economic evaluation’ is based on the single-arm study described above (Iemmi et al, 2015). In addition to the WEMWBS and HPLP II, participants also completed the Client Service Receipt Inventory (CSRI) to measure service use.
The cost of the intervention is estimated to be £894 per participant, though it isn’t clear how this was estimated. This cost is offset by decreases in service use over the period.
Results are presented for two subsamples based on the completeness of data; one shows a small increase in costs and one shows a decrease.
There is no primary analysis to follow, and the results lack clarity.
It’s not clear how the researchers have estimated the costs of the intervention.
Strengths and limitations
The development and delivery of the programme is impressive. The approach adopted was firmly grounded in lived experience and service users led the content, format and delivery of the programme.
Peers leading self-management and peer support approaches – all good stuff. And to run the course over 60 times with over 600 participants strongly suggests that it’s feasible to do and highly acceptable to participants. But (and it’s a large ‘but’) we are curious about the choice of evaluation design for this project as so many questions are left unanswered by these papers.
Another elf (Hughes, 2014) has argued that we need to better define and categorise peer support, so that we know what it is, then test it and seek to understand what is the active ingredient that produces good benefit. These papers do a good job at describing the ‘type’ of peer support, but don’t help us with a robust test or illuminate what it was about the programme that (possibly) produced the good benefit.
The study does not include a control group, let alone randomisation.
An economic evaluation, by definition, considers two or more possible activities; this study does not. It is not possible to attribute any change in wellbeing and health promoting lifestyle or the change in costs to the intervention. The authors are wrong to describe part of their study as an ‘economic evaluation’. Much of the reporting in the manuscript is misleading, and groundless claims are made.
While the authors acknowledge some limitations, they spend quite a bit of time speculating what might lay behind the increases in wellbeing and health promoting behaviour measured over time. But actually, we don’t know because the experiences and views of the participants weren’t captured.
Reading these papers, you’re left wondering what the participants thought of the intervention and how they perceived it may have helped with their wellbeing and health promoting lifestyle behaviours. If this study had included a controlled comparison of outcomes, and some understanding of the participants’ experiences and perceptions, it would have been a really useful contribution to the evidence base. So a missed opportunity, perhaps.
We wanted to know more about what the participants thought of the intervention.
Summary
While the quality of this evidence isn’t as strong as would usually appear in the woodland, the importance of the topic is, and we hope this blog continues the debate.
By publishing these papers, the authors have met their aims to develop and deliver a coherent, testable intervention for self-management for people with severe mental illness, but unfortunately, not their third aim of evaluating it.
Usefully, we’ve learnt that peer-led self-management programmes co-produced with people who’ve accessed services, are feasible to design and deliver and are acceptable to participants.
We’ve also learnt that people who’ve taken part have reported a simultaneous improvement in their wellbeing and health promoting behaviour and reported a reduction in costs.
However, the research design means that we can’t be sure whether this positive finding was consequential or coincidental.
This study shows that coproducing peer-led self-management programmes with service users is feasible and acceptable to participants.
Links
Primary paper
Cyhlarova E, Crepaz-Keay D, Reeves R, Morgan K, Iemmi V, Knapp M. (2015) An evaluation of peer-led self-management training for people with severe psychiatric diagnoses. The Journal of Mental Health Training, Education and Practice, Vol. 10 Iss: 1, pp.3 – 13 [Abstract]
Other references
Crepaz‐Keay D, Cyhlarova E. (2012) A new self‐management intervention for people with severe psychiatric diagnoses. The Journal of Mental Health Training, Education and Practice, Vol. 7 Iss: 2, pp.89 – 94 [Abstract]
Iemmi V, Crepaz-Keay D, Cyhlarova E, Knapp M. (2015) Peer-led self-management for people with severe mental disorders: an economic evaluation. The Journal of Mental Health Training, Education and Practice, Vol. 10 Iss: 1, pp.14 – 25 [Abstract]
Hughes L. (2014) No support for peer support. The Mental Elf, 25 Apr 2014.
All three papers are also hosted on Dr David Crepaz-Keay’s personal archive http://www.peer-support.info/
