In recent years there has been a (welcome) trend towards research being carried out ‘with’ or ‘by’ members of the public rather than ‘to’, ‘about’ or ‘for’ them (NIHR). Whilst there is movement to standardise this across all health research (for instance many funders now require evidence of patient and public involvement), disparities remains in the types and stages of research where the public are involved.
Although patient and public involvement is advocated at all stages of the research cycle (NIHR), there is seldom lived experience input into, for instance, the selection of outcome measures, item development and establishing comprehensibility (Wiering, de Boer & Delnoij, 2017). This gap poses a fundamental challenge to health research. Why should researchers, funders or policymakers alone determine which outcomes are most important? This approach risks leading us astray – how can we be sure that we are prioritising the right questions and measuring the right outcomes? For instance, instead of solely assessing whether a new talking therapy reduces clinical symptoms of depression, might there be deeper, more fundamental aspects of people’s experiences that we should be investigating?
This blog delves into this specific issue, exploring a systematic review by Molloy and colleagues (2025), published today in The Lancet Psychiatry, which aimed to “identify mental health outcome measures currently in use that meet a strict definition of being co-developed.” (p. 2).
Is our current approach to choosing outcome measures leading us down the wrong path?
Methods
The authors (including three of whom who identify as having lived experience of mental ill health), conducted a systematic review of papers describing the development of new Patient Reported Outcome Measures (PROMs) using quantitative, qualitative or mixed methods. Searches were conducted across MEDLINE, Web of Science, Scopus, PsycINFO and Embase as well as a grey literature search. Studies were excluded if they: were not primarily in a mental health population or condition; did not describe the original development or psychometric testing and used co-developed PROMs as an endpoint to collect data. All papers were screened by two authors independently at both title and abstract and full paper stages.
Included measures were rated on a scale of one to three where one meant that a lived experience group was consulted, two denoted a service user-researcher was present in the team and three points were awarded where both things were evident. No score was given if there was no reporting of lived experience involvement. Ratings were given at each stage of measure co-development including generation of items, feedback on items, psychometric testing and write-up. The GRIPP2-SF (Staniszewska et al., 2017) was used to assess the quality of reporting of lived experience involvement in the included papers.
Results
A total of 34 papers, describing 23 PROMs were eligible for inclusion in the review. The majority of measures were mental disorder specific and developed with adults in the UK. The average score of papers on the GRIPP-2 SF checklist was 9 (good) with only two measures graded as excellent. In general, the quantity of lived experience involvement was highest in the development of items and lowest in the psychometric testing phase.
For the generation of items, service-user researchers were the most common type of involvement (11 PROMs), though some PROMs also included advisory groups (n=4), steering committees (n=2), reference groups (n=2) or consultant groups (n=1). Eight PROMs did not specify particular lived experience groups, but described involvement via focus groups or qualitative interviews.
Feedback on items was gathered via focus groups, interviews and Delphi exercises. For 11 PROMs, such feedback led to reworded items, removing items and formatting of final measures.
Thirteen PROMs involved people with lived experience in tasks beyond completing measures. Other ways that people with lived experience were involved in the psychometric testing phase included designing feedback forms, selecting comparison measures, reviewing analysis results and refining items.
At least one person with lived experience was involved in the write-up and dissemination for 14 PROMs.
This review suggests that the UK is leading the way in co-developing outcome measures in mental health research.
Conclusions
Traditional methods of measure development often exclude people with lived experience or involve them only minimally—for example, by including service users in item generation or an acceptability focus group, but with no further engagement before or after these activities. This review has shown, however, that it is both possible, and beneficial, to meaningfully collaborate with people with lived experience in the development of PROMs.
It is both possible, and beneficial, to meaningfully collaborate with people with lived experience in the development of patient reported outcome measures.
Strengths and limitations
This was a well conducted systematic review that showed thorough and rigorous commitment to methodology and was well executed. The involvement of those with lived experience in the research team, right from conception of the study, is to be commended; this paper has a clear focus on those with lived experience and it is wonderful to see.
However, it would be good to see some more justification from the authors about their chosen scoring criteria for ranking the level of involvement in included papers. Is a service-user researcher necessarily always a ‘better’ form of involvement than a lived experience group? Perhaps so, but it would be good to know a bit more about what led to the decision, and indeed the extent to which lived experience itself played a part in this decision.
Overall though, the main limitations of this review exist in the body of research being reviewed in this study; it is unfortunate to see so few of the included papers scoring ‘excellent’ on the GRIPP-2 SF measure.
Are service-user researchers necessarily a ‘better’ form of involvement than lived experience groups? Possibly, but more justification is warranted.
Implications for practice
Personally, I feel that the conversation around user outcomes is not a new one. Indeed, I am reminded of this excellent paper published back in 2017 by one of our fantastic Elves, Alison Faulkner, in which she states:
There are few explorations of issues of identity, considerations of mad culture or the impact of treatments/services on user-defined outcomes rather than those that are service or symptom related. Curiously, this contrasts significantly with Disability Studies where cross-disciplinary work including and carried out by people with disabilities, plus a respect for first-person narratives, results in a respect for the knowledge originating from ‘experts by experience’ – in this case disabled people. (Faulkner, 2017)
I am glad to see that we have begun to move into an era where we are now quantifying and reviewing the lay of the land in this field, but I can’t help but feel more is needed. Even at the most basic level I would implore researchers to consider this; have you ever asked people with lived experience what they think the main outcomes of your study should be? I began doing this in my PhD (Hemming et al., 2021; Nedoma, 2021) and continue to do so in every study I get funding for. The reaction I get from other researchers is always one of surprise and congratulations – but I would argue this should be commonplace by now and should not come as a surprise to other researchers.
Have you ever asked people with lived experience what they think the main outcomes of your study should be?
Statement of interests
The author has no interests to declare.
Links
Primary paper
Molloy, N. Kilcoyne, I., Belcher, H. & Wykes, T. (2025). Exploring the involvement of people with lived experience of mental health disorders in co-developing outcome measures: a systematic review. The Lancet Psychiatry. 10.1016/S2215-0366(24)00376-6
Other references
Faulkner, A. (2017). Survivor research and Mad Studies: the role and value of experiential knowledge in mental health research. Disability & Society, 32(4), 500-520.
Hemming, L., Shaw, J., Haddock, G., Carter, L. A., & Pratt, D. (2021). A cross-sectional study investigating the relationship between alexithymia and suicide, violence, and dual harm in male prisoners. Frontiers in psychiatry, 12, 670863.
Nedoma, R. Alexithymia and suicide, violence, and dual harm in male prisoners. The Mental Elf, September 2021.
Staniszewska, S., Brett, J., Simera, I., Seers, K., Mockford, C., Goodlad, S., … & Tysall, C. (2017). GRIPP2 reporting checklists: tools to improve reporting of patient and public involvement in research. bmj, 358.
Wiering, B., de Boer, D., & Delnoij, D. (2017). Patient involvement in the development of patient‐reported outcome measures: a scoping review. Health Expectations, 20(1), 11-23.
Photo credits
- Photo by Elie M on Unsplash
- Photo by Jens Lelie on Unsplash
- Photo by Chris Boland on Unsplash
- Photo by Clay Banks on Unsplash
- Photo by Florian Schmetz on Unsplash
- Photo by Emily Morter on Unsplash
