We are living through what Erik Brynjolfsson and Andrew McAfee call The Second Machine Age, where digital tools will surpass physical ones and replace, rather than complement, human labour. This future is one that is faced with ambivalence by both people who experience mental health difficulties and those that provide services intended to support and treat them.

Digital interventions in mental health have long had the status of a ‘King Over the Water’; promising to remake the world and redefine everything about mental health treatment, but never quite arriving.  

Health systems across the global north have tended to view digital interventions purely in terms of increasing access to psychologically-informed support or treatment or reducing costs. Innovators tend to see such developments as ‘disruptions’ that might change at a fundamental level the models and systems by which mental health support is delivered, with potential to even make such services obsolete. Both look to scale as the goal, either through organisational and systematic implementation or through consumer choice. Both have an ambition to go big or go home; often in contrast to their actual number of users.

To date, most of the heat in this new digital industrial revolution has focused upon either prevention or upon the points at which mental illness crosses over with common challenges of living, such as mild depression or anxiety. In the first systematic review of digital mental health implementation for interventions aimed at those with a diagnosis of psychosis or bipolar disorder, published in The Lancet Psychiatry this week, Golnar Aref-Adib and colleagues ask ‘what do we know about implementing these digital wonders?’ The answer, it turns out for people with those conditions using digital interventions, is that we don’t know very much at all. Mainly because we’ve not actually tried very much.

Methods

“Factors affecting implementation  of digital health interventions for people with psychosis or bipolar disorder, and their family and friends” is a systematic review developed in line with Cochrane Collaboration guidance on conducting reviews and follows the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) guidelines in reporting its findings. The protocol for selecting studies was registered with the PROSPERO International prospective register of systematic reviews. The review searched for studies that included adult service users of digital interventions with a diagnosis of psychosis or bipolar disorder established by any recognised diagnostic method and for studies that focused on the friends and families of people with these diagnoses.

The reviewers did not conduct a quality appraisal of the studies included. They justified this by saying that the “published literature in this field is in its early stages; we wanted to be inclusive and we were not examining effect sizes”.

The review used a framework analysis method to look at the factors in implementation using data analysis tool NVivo 11. This used the Consolidated Framework for Implementation Research (CFIR), a “menu of constructs that have been associated with effective implementation” of innovations, to look at a number of the major constructs in actually putting something into action. These include: intervention characteristics such as relative advantage, complexity and cost; outer settings such as needs of patient, incentives and policies; inner settings such as networks, resources or the climate within the context the innovation is being implemented; and the actual characteristics of those involved.

The review included any studies published after 1st January 1995 with data (including individual, organisational or system related) relating to factors that affected the implementation of specific digital interventions for people with a diagnosis of psychosis or bipolar disorder. The review excluded screening tools, interventions that did not directly engage those with the diagnosis or their family and friends and those that were not yet in use, even in a research setting. It also excluded studies that only included implementation as a topic in their discussion and those that were only available as abstracts.

Results

The review identified 3,359 unique citations. After reviewing titles and abstracts, 3,026 were excluded. Of the remaining 333 studies, 26 met the inclusion criteria. All of the papers were written in English. Of the included studies, seven included those with diagnoses inclusive of psychosis, seven those with diagnosis of bipolar and the rest included both. No studies were found that addressed implementation of interventions with family and friends.

Seventeen of the studies were published between 2016 and 2017. The most common studies were feasibility and acceptance trials (10 of the 26). Eight looked specifically at implementation. Five were pilot studies and two were randomised controlled trials.  

Twenty of the studies discussed digital interventions that had support from staff or peers in their delivery. Only two of the studies used an implementation plan and none reported using implementation theory.

The authors summarise their findings in the discussion of the paper. They note that in the 26 papers:

the evidence… regarding organisational and process factors that affect implementation for digital interventions was scarce. 

Of the studies they identified, few were really detailed studies of implementation. They remark that “most of the studies identified were preliminary evaluations of the acceptability and feasibility of digital health interventions for this population rather than implementation studies. Further, none of the studies used implementation theory and we retrospectively organised their findings into the CFIR.”

They concluded that this lack of implementation science is because:

digital interventions for psychosis or bipolar disorder are not as established as those for physical or common mental health problems.

They also note that, contrary to other reviews, the evidence for external policies and incentives being important to the implementation of digital interventions was scare in their review.

Conclusions

The review found that all of the studies included identified multiples factors that were barriers to implementation but with no single factor being identified as either a key barrier or facilitator. The authors state:

The majority of factors for effective implementation of digital interventions were centred at the level of the individual or the intervention. Digital health intervention users were more likely to complete an intervention if facilitated by staff or peer support, and if the intervention had been proposed by a staff member who found it useful.

In the studies the review did identify, there was no consistent definition of what engagement with a digital intervention meant and no data in the studies reported on the representativeness of the study samples, meaning it is difficult to generalise across studies. The authors acknowledge that the diversity of the interventions themselves varied, as did the setting of their implementation, again suggesting that it is difficult to draw meaningful comparisons.

Implications for practice

Across the studies included in the review, barriers to successful implementation included:

• A lack of belief from both professionals and service users that there was an advantage in the intervention relative to other existing means of support or interaction
• Fears about cybersecurity and privacy
• A lack of interoperability between digital interventions and existing systems within services
• The requirement for upfront investment of resources and staff time before any benefits might be seen
• “Disparity between the IT skills required for the intervention and those skills the users and the staff have”
• The lack of actual digital or physical infrastructure (such as wifi, computers, printers) to actually implement the intervention.  

The review found that:

the complexity of the digital intervention was a barrier for people with psychiatric symptoms, low premorbid intelligence quotient, or low IT skills, as these often resulted in difficulty concentrating, engaging, and completing interventions.

The authors remark:

Female gender and being white were associated with more successful completion of interventions.

Discussing individual diagnoses the authors say:

People with bipolar disorder mentioned concerns on how digital interventions could affect their mental health, and a minority of people with psychosis became paranoid or had symptom exacerbation.

They also noted that:

high completion outcomes were achieved for smartphone interventions in users with a high functional assessment short test score, more years of smartphone usage, and high premorbid verbal intelligence quotient.

Finally the authors say:

The accessibility and adaptability of digital interventions were key facilitators, but their cost was a barrier. Although evidence to support the case for digital interventions making long-term savings is available, the upfront costs for developing interventions and the ongoing delivery costs are likely to be important factors in services transitioning to more digital services.

Strengths and limitations

The strength of the systematic review is that it summarises what we know about the implementation of digital interventions for people with a diagnosis of psychosis or bipolar disorder. The limitation is that we don’t actually know very much. This review focuses on implementation, and as a result tells us very little about what was actually being implemented; a vital element in discussing a non-standardised intervention.

Discussion

The study of digital interventions for mental health and the mitigation of mental illness remains in its infancy, often lacking even the tools to understand the thing upon which its lens in focused. Digital interventions are designed artifacts which are deployed using particular devices, which in turn might be interacted with in a variety of conditions in a multiplicity of settings. It is an old joke that bad software designers blame the user, but one that the findings of the review suggest has not gone out of fashion in discussing digital mental health interventions, with studies suggesting that implementation fails when the user is too ill, too inexperienced or not clever enough to understand the intervention.

Without a granularity in understanding and contrasting the digital interventions themselves; the conclusion of the review is a broad ‘some people liked them, some people didn’t and the ones that people liked were easier to implement and tended to have some positive results but only when staff liked them, too’.  What is missing is the dimension of understanding the intervention as a level of implementation in itself.

Unlike medical trials: who the user is, what their expectations are and how those expectations manifest as preferences deeply affects their response to a digital intervention. In turn, the efficacy of the intervention depends greatly on how the user experiences the intervention as a whole. A digital intervention is an experience situated within someone’s life as much as it is a quantifiable medical phenomena. In medical terms, an injection can be a more or less pleasant experience depending on where or how you are administered it and for what reasons; but that experience does not alter the chemical composition of the solution forced down the barrel. With digital interventions, the application, device or program is solution, the barrel, the needle and the context rolled into one experience. From a systematic review based on looking for data which influenced implementation, it is impossible to ascertain what the interventions had in common in terms of presentation and user experience.

The review found there were great differences between studies as to who responded to the interventions: “White users were found to be more engaged with a mobile intervention for individuals with schizophreniform disorder than Hispanic and African American users. In two studies, young people (<30 years of  age) with psychosis or bipolar disorder were less likely to engage and complete the digital intervention than those who were older. In a study with a mixed population, patients with a vocational education had more successful educational sessions than those without vocational  education. However, in other studies for individuals with bipolar disorder, no statistically significant correlations were found between engagement with digital intervention and age or education level.” Without a strong understanding of what those interventions were; or indeed when and where they were delivered to people in what conditions, it is hard to understand what actually occurred.

It is notable that most of the small number of studies came from the last two years; but even then it is clear that only a tiny amount of work is being done to actually implement digital interventions around psychosis and bipolar depression. There is a sense in which the systematic review in question reenacts the promise of digital interventions over their actual current position. This is not a fault of the review; but of the overall condition of the industry of creating and iterating mental health interventions as a whole. At present, clinical interest in such interventions is often not backed by investment and investment in the wider market for digital mental health interventions has little incentive to chase anything other than common mental health difficulties where the market potential for sales direct to the end user is greater.

It is perhaps too early to focus upon implementation of digital interventions for bipolar and psychosis, despite the pressing need to do better for people experiencing the difficulties that comes with those diagnoses. From this review, it might be concluded that having something to actually implement and understanding how to make it work for people as well as possible is a more pressing task than trying to understand how such an intervention, were it to arrive, might be rolled out at scale.

For people living with the negative experiences that we diagnose as bipolar disorder and psychosis, it seems that that digital King Over the Water will not be arriving any time soon.

Links

Primary paper

Aref-Adib G, McCloud T, Ross J, O’Hanlon P, Appleton V, Rowe R, Murray E, Johnson S, Lobban F. (2018) Factors affecting implementation of digital health interventions for people with psychosis or bipolar disorder, and their family and friends: a systematic review. The Lancet Psychiatry December 03, 2018 DOI: https://doi.org/10.1016/S2215-0366(18)30302-X

Photo credits

• Photo by Bogdan Glisik on Unsplash
• Photo by Ali Yahya on Unsplash
• Photo by Toa Heftiba on Unsplash
• Photo by rawpixel on Unsplash