In 2009, mental health care being delivered through different arrangements of health and social services in four boroughs of South London and Maudsley (SLaM) was reorganised. Instead of being delivered by district, SLaM set out to offer services based on mental health difficulty and/or age group. Services would offer the same care pathways across the entire area through seven Clinical Academic Groups (CAGs).

The intended benefits of Clinical Academic Groups included:

• Equality of provision across geographical areas
• A closer working relationship between academia and clinical activity
• An alignment of services with the increasing interest on maps or patient flows through guideline-based care pathways.

This detailed report is written mainly by clinicians and academics working in SLaM who evaluated one of the CAGs involved in the project (the Mood, Anxiety and Personality CAG) over a five year period. This means that there is a lack of independence from the process of reorganisation for those appraising that reorganisation.

Service reorganisation

SLaM as an NHS Trust was only ten years old in 2009 and had only been a Foundation Trust for three years. This apparently rapid wholesale service reorganisation will no doubt be of little surprise to commissioners, although it may seem rather alarming to the general public. SLaM covered the boroughs of Croydon, Lewisham, Lambeth and Southwark, with a total population of over 1.2 million people.

The first stage of this process was the agglomeration of three NHS Trusts (SLaM, Guy’s and St Thomas’ and Kings College Hospital) and two university institutes (the Institute of Psychiatry and King’s College London School of Medicine) to form the King’s Health Partners Academic Health Science Centre (KHP AHSC). Seven CAGs were then created as stand-alone, self-managed organisations under the umbrella of the KHP AHSC. These were supposed to be clinically driven and to result in more targeted, consistent, recovery-oriented services, innovation in care and greater integration of health and social care.

However, the major significance of the CAGs appears to have been their impact on structures and relationships, as they required the introduction of completely new managerial systems. Those entrusted with managerial roles in a CAG held the responsibilities for clinical, financial and research streams and outcomes.

KHP devoted two small project teams to help trusts to develop CAGs:

…it was intended that the restructuring into CAGs would also lead directly to improvements in clinical services.(p.18).

However, the research team found little evidence of this intended innovation, or indeed of service line management or of integration of various types of health and social care into a recovery-oriented model.

Part of the reason for this lack of clinical service development may have been that the services in the four initially separate boroughs were very diverse. At the same time, there was a shared sense of difficulty across all four boroughs in altering the relationship between primary and secondary care, and staff working in all four boroughs wanted to try and make changes to this relationship. The extent to which these similarities and differences were considered during the planning process and were used to shape the restructuring process remains unclear; it seems as though these factors were recognised but not necessarily acted on, and so a potentially important lesson for the future would be to make sure such elements are duly reflected on in the planning process.

It seemed that staff at higher managerial level and staff at the service level had different motivations for supporting the implementation of the CAG model, and that there was some cynicism about the ‘academic’ KHP element. For managers, seeing Payment by Results coming on the horizon was evidently a major motivator, but this was perhaps not a key factor for clinicians in terms of service design. There were greater concerns for clinicians in terms of the three adult CAGs and the artificial segregation these created between mental health conditions; concerns shared by service users. Understanding the motivations and priorities of staff at all levels involved in an implementation can be helpful in promoting engagement and in sustaining implementation in the longer term.

The problem with really understanding what went on in implementing the CAG model comes down to the methods employed by the research team.

 

Research method: A realist evaluation

The aims of the evaluation were to:

1. Consider how the programme of reorganisation was conceptualised
2. Appraise the impact of the reorganisation in terms of activity and quality changes
3. Consider lessons that could be learnt from the experience

A realist evaluation approach is particularly pertinent to addressing the third of these objectives, since realist approaches look to achieve generalisable findings for practical application.

Qualitative methods

The qualitative work undertaken is oddly and seemingly poorly constructed, starting with an unusual sample of a very limited number of staff and service users. Semi-structured group, pair and individual interviews were conducted with 13 service users and 6 staff within the first four months of the reorganisation. Three members of staff were then re-interviewed 27 months after the start of the restructuring process.

It is worth just stopping here and considering this as a design for evaluating the impressions of a service reorganisation of such a scale, although it is hard to know where to begin. As far as I can tell, no logic for the sampling approach, frame or size is presented in the study report. Service users were found by a PPI representative, but how they were selected or their possible representativeness of what must be close to one million potential adult service users is unclear. The staff who were interviewed were targeted as workers at the managerial level, but this means that no information was gathered from any non-managerial clinicians about their impressions of the new service.

The problems with the qualitative work continue from here, with: a range of data from interviews, groups and documents; two time points for data collection that seem very poorly chosen for the aims of the study (interviewing people about the CAGs within three months of the start of the implementation seems incredibly premature, and only interviewing three members of staff at time two about the implementation would reveal very little); and a lack of a theoretical model or analytic framework based in implementation science for data analysis. As such, the qualitative section feels like a description from the perspectives of a cluster of unrepresentative people, which therefore makes it rather unreliable and uninformative.

The quantitative analysis seems to have been on firmer ground, although some of the measurements used seem somewhat mismatched to the categories they are representing, such as using waiting times as the measure of ‘person-centeredness’ of the CAGs. However, data was collected both before and during the implementation of the MAP CAG to assess the demographic profile, clinical outcomes scores (HONOS), A&E attendance and self-harm levels for each borough individually, in order to see how the CAG service compared to each region’s previous service offering. Overall, there was little variation for any individual borough from pre-MAP CAG to post-MAP CAG and it is of most pertinence that clinical effectiveness did not seem to alter with the introduction of the new service.

The report provides some comparative evidence of other reorganisations taking place in London and how these areas had reordered service line management, which may be of interest to those involved in commissioning similar services.

 

Outcomes

The striking finding of the evaluation was the virtual lack of clinical impact resulting from the service reorganisation. Whilst clinical activity levels actually appeared to be reduced, patients seemed generally unaware of any changes and care pathways were not really developed. Since the four boroughs started with different activity and cost levels, it is unsurprising that different directions and scales of change were found for each region, but it is somewhat alarming that the enormous work of creating new management structures and the incorporation of academic expertise into clinical streams did not improve patient safety or clinical effectiveness overall.

The changes that were perceived by the interviewees were in the systemic changes. Staff had wanted things to be reorganised, and so perhaps it is not surprising that they felt the changes were beneficial. Before the reorganisation, there was a high level of concern amongst staff about financial instability and so the achievement of financial stability with the creation of CAGs was no doubt a significant source of relief.

How was it that clinical pathways did not materialise as intended i.e. as NICE guideline pathways?

There appear to be at least two possible answers to this important question. Firstly, the CAGs initially looked at using the clinical pathways model as a means of developing generic pathways that described managerial processes and system activities. These ‘high-level’ service pathways were seen as useful, giving managers a framework from which to make sense of the reorganisation, and as such it may be that staff became focused on what was working well, rather than on taking on the harder work of expanding these into clinical care. Staff found that prescribed pathways did not map onto the more complicated real world of mental health care.

Secondly, whilst managerial units were totally reorganised in the CAG system, pre-CAG clinical groups still existed, and it was much harder to get these to fit into the processes dictated by care pathways. Tackling the restructuring of treatment and care teams would be a much larger and more daunting proposition, and it may be one that is feasible with more time, but the momentum behind care pathways may be much decreased by this point in time, both for the CAGs in terms of service fatigue, and in the wider context with the waning enthusiasm for Map of Medicine.

What is hinted at by the stage two respondents is that the MAP CAG appears to be developing its own brand of care pathways, and it may be that these grow out of the reorganisation in a more natural way that fit well with the CAG rearrangement. Whether these have a clinical impact will have to be evaluated over time; hopefully the service has evaluation measures in place.

For service users, the overriding impression was one of not being involved in neither the design, nor the promotion, nor the evaluation of the CAGs. Whilst we are looking retrospectively at patient involvement decisions taken five years ago, it is disappointing to see how excluded service users appeared to have been from such a major service reconfiguration.

We need to bear in mind that this report is a wide ranging and highly interpretive body of work based on a relatively small amount of data from a qualitative standpoint. This is particularly true for some interesting findings drawn from the interviews at the second timepoint, when only three managerial respondents were interviewed. There was a sense that the CAG reorganisation had worked well for the child and older adult services but not for adult mental health, as the different arrangements in each borough, and the associated relationships between borough teams and their commissioners, were felt to be too disparate to be united in the CAG model. These ideas could have been developed in more detail if more voices had been sought, particularly from clinical staff working in the services. Commissioners would also have made interesting research participants.

The research team focused on one CAG; although it is suggested that the MAP CAG is representative of all the CAGs, it is not possible to verify this assumption, and indeed respondents suggest quite a different picture for the non-adult mental health CAGs.

The proximity of centres of academic reknown such as the Institute of Psychiatry and the existence of a regional National and Specialist Directorate may be contextual factors unique to SLaM that influenced the reorganisation in a way that might be unlikely to be seen in other localities. This poses a challenge to the generalisability of this ‘realist evaluation’, although the qualitative sampling approach had perhaps already undermined the generalisability anyway.

Put not your trust in new leaders, better systems, new organisations or regulatory reorganisation. They may well be good and necessary, but will to some degree fail.

– Justin Welby

Links

Primary paper

Tulloch AD, Soper B, Gorzig A, Pettit S, Koeser L Polling C, Watson A, Khondoker M, Rose D, McCrone P, Tylee A, Thornicroft G. (2016). Management by geographical area or management specialised by disorder? A mixed-methods evaluation of the effects of an organisational intervention on secondary mental health care for common mental disorder. Health Services and Delivery Research, 4(9). DOI: http://dx.doi.org/10.3310/hsdr04090

Photo credits

• emilydickinsonridesabmx (CC BY 2.0)
• Pixabay (CC0 public domain)