It is rapidly becoming clear even to those most resistant to digital technology that our digital lives have an impact on our mental health.

Whether it is in understanding the complex digital lives that digital natives lead, recognising the need to appreciate people’s digital lives in the context of care and treatment, or creating public mental health approaches to digital lives, we need a framework with which to understand and contextualise digital technology and mental health.

Every week another research study is interpreted beneath a sensational headline, or analysed by this blog.

Evidence, or the appearance of an evidence base is sought as safe harbour in a stormy sea when clinical staff, commissioners and policy makers come across a challenge which may be beyond their comfort zone or experience. Amid the increasing recognition that digital will not go away is the increasing thirst for quality data and research studies to help in framing responses.

A paper that promises a systematic review and meta-analysis of evidence on social media interventions for complex mental health problems is likely to be fallen on by hungry lions receiving an emailed contents page. It is therefore critical that such a review presents a strong precis of the evidence, and a guide for what this indicates.

Professor Välimäki and colleagues promise much, with a systematic review and meta-analysis on the effectiveness of social media interventions in schizophrenia.

Sadly, this review again runs into a lack of ‘high quality’ studies in this field, and therefore becomes a discussion of what might be, and where the gaps may lie. Because the reviewers did draw some conclusions, based on only three studies, at first glance, this seems to say ‘review, meta-analysis, social media didn’t help much’.

I remain concerned that rather than create incentives for the funding and publication of good quality studies in this area, reports like this may place digital mental health strategies back in the long grass when clinicians in the UK already face so many challenges, particularly around shifting resources, and on risk enablement.

Methods

A robust systematic review methodology was undertaken, in line with well-established protocols and data extraction techniques. Ten databases were searched, bringing together searches for schizophrenia and related topic with a wide range of social media terms including most of the popular brand platforms (excluding Tumblr), and a range of generic terms like “blogging”, “wiki” etc.

The inclusion criteria included a requirement for randomised controlled trials (RCTs) published in English, in peer reviewed journals, before June 2015, with abstracts.

Specific inclusion criteria of interest included:

• A requirement for studies to relate to interactivity (defined as user-to-user contact by people with schizophrenia spectrum disorders)
• That action/intervention was within a health care domain and targeted adults with various schizophrenia spectrum diagnoses. If a study included adolescents, it was included only if the mean age of the participants was over 30 years.

The definition of social media used was broad, and included interventions including:

• Collaborative projects (e.g. user-generated content, content communities, content sharing, and social online networking sites)
• Social networking
• Interventions involving Facebook, Twitter, YouTube, Instagram, blogs, Wiki, chats, the Internet, or Web 2.0.

Of 1,023 records identified, 727 abstracts were screened. Of these 11 full text papers were assessed for eligibility. 8 of these were then excluded for various reasons, the principal one being that they did not use social media as defined:

• 3 papers were quantitatively analysed
• 2 papers were included in the meta-analysis.

A robust meta-analysis methodology was used, based on the Cochrane Collaboration framework (McMillan S, 2002). The aim was to ascertain to whether the evidence supported social media as an effective intervention in people with schizophrenia.

Results

Two papers were included in the meta-analysis. The two studies took different approaches to providing a managed, custom built social media opportunity for people with schizophrenia, and their supporters.

Rotondi et al (2005) developed a psychoeducation platform (SOAR) that aimed to provide multi-family psychoeducational therapy to people with schizophrenia/schizoaffective disorders and their supporters. This was delivered via home PCs (supplied where participants did not have them). There was a 4-hour orientation session to teach people how to use the SOAR website. Participants were recruited from inpatient and community mental health settings, and must have had ≥1 psychiatric admission or emergency department attendance within the previous 2 years. The control group were given ‘treatment as usual’, though this was not defined. 31 participants were recruited (mean age 38), with 1 person (3%) leaving the study early.

Kaplan et al (2011) recruited 300 individuals via an open recruitment method, which included social media and a public advertisement to participate in unmoderated, unstructured online peer support. Participants had to be US based, have computer access, and had to have not used online peer support in the last year. They were randomised to either a Peer Bulletin Board, Peer Listserv or waitlist control group. Those in the waitlist control group were asked to refrain from all other online peer support for the duration of their participation (1 year). 41 participants (14%) withdrew early from the study.

A range of outcome measures were used across the two studies to measure effect on positive symptoms, self-management, stress, quality of life etc. In addition Kaplan et al (2015) used the Online Group Questionnaire (OGQ) to track participants’ feelings about the group dynamic, their participation, and the relevance of discussion to them.

Rotondi et al (2005) measured engagement in the SOAR platform, and the use of the resource. Participants spent an average of 46 hours on the site, with around 3,000 page views. They asked an average of 113 questions, and spent 1,643 minutes reading articles. People with schizophrenia were active in 3,300 therapeutic sessions during the study.

All in all, 11,105 messages were sent by people with schizophrenia during the Kaplan et al (2011) study. Investigators categorised participants into ‘high dose’ and ‘low dose’ groups. High dose participation was defined as having read messages at least weekly, and having sent at least 5 messages in the study. They also grouped participants by OGQ score; finding that those with positive online experiences were significantly more distressed than those in the less positive experience group, i.e. that those who got the most out of the online activity were also the most distressed.

The meta-analysis was unable to compare directly the effect on symptoms after 6 months as the Rotondi et al (2005) study did not provide relevant information, but in the Kaplan et al (2011) study there was some improvement in the social media group at 6 months. The Rotondi et al (2005) study showed some positive effect on self-reported stress in the social media group in the same timeframe.

In wider social outcomes at 6 months, both studies indicated a slight improvement for social support in the treatment and usual/control groups at 6 months ((P=.03, median 0.22, 95% CI 0.02 to 0.42). This was the only area where a direct comparison was possible between the studies, and the effect was small.

Kaplan et al further examined self-management, and quality of life at 4 and 12 months. They found that self-management was slightly more effective at 4 and 12 months in the treatment as usual group. Conversely, the self-reported Quality of Life scores were significantly higher in the social media groups in the same time periods.

Conclusions

The reviewers concluded:

Our findings suggest the effects of social media interventions are largely unknown. Use of social media forums is ubiquitous and increasing, but the relation between social media and mental health is complex, not well understood, and potentially detrimental. Thus, we suggest that this is reason enough to support further investigation. Emerging evidence suggests that online social networking can be related to major mental health problems such as depression, but at the same time online and mobile-based interventions for people with psychosis seem to improve depression. Given the constant increase of social networking sites, it is understandable that recent studies have identified the need for exploring such sites and mental health. Future research should comprehensively assess social media use for people with mental illness to determine the impact of mental well-being for social media use, as well as its risks (Välimäki M, 2016).

Strengths and limitations

The paucity of quality studies in this area brought the reviewers to the point where of 1,043 identified records only 2 trials of moderate quality could be included in the meta-analysis.

It is exceedingly hard to draw conclusions from a meta-analysis of two moderate quality trials. More evidence is clearly needed before the conclusions can be regarded with any certainty. Furthermore, are number of concerns about this specific review and both of the included papers, especially in relation their potential to inform contemporary policy and commissioning.

The inclusion criteria for the study seem extremely stringent, particularly in relation to the requirement for interventions to use social media, in clinical environments or contexts, with adults with a mean age >30. It is unsurprising that few papers were considered, given the adoption challenges of social media in clinical contexts and the assumption that many digital innovations aren’t sufficiently mature or stable to have been subjected to RCTs and accepted for publication. There aren’t many peer reviewed RCTs that use native (i.e. not experimentally designed) social media platforms in health care settings to target adults with various schizophrenia spectrum conditions. The decision not to include studies with adolescents or where the mean age of mixed samples was <30 years also mitigates against studies that work with the very population most likely to use social media, and to benefit from services understanding their digital lives.

Välimäki et al hinge their paper on a broad definition of social media. This is bold, but neither of the eligible studies included is an RCT of what most people would recognise as social media in 2016 (i.e. Facebook, Twitter etc). Listservs and Bulletin Boards were identified as a medium of online peer support in one of the included trials (Kaplan et al, 2010). In the other study, a custom build website was used to provide psychoeducation to people with schizophrenia, and their carers (Rotondi et al, 2010). Both of these methods of digital engagement are useful, and we need an evidence base for their use. We see ‘closed’ social media platforms such as SHaRON developed in house within the NHS to meet a defined clinical need (in this case a need to give patients with eating disorders a space to interact online without the risks they’d otherwise have to face).

At the same time we must recognise that decisions about commissioning services for a 2020 NHS cannot be based on the evidence gathered about the use of what is essentially 1990s technology. Listservs, Bulletin Boards and Psychoeducation via static PCs on dial-up have been superseded in most people’s digital lives by multi-channel, multimedia tools that include both mental health specific and repurposed general tools, and peer support from a range of peer connections in a range of online and offline spaces.

We need to find ways of studying the contemporary digital ecosystem in which most digital natives (most often those born post-1987) and digital refugees (those who find a peer connection and relief from isolation and poor quality of life in enduring online social capital) find themselves. Kaplan et al (2011) describe as ‘high dose’ users as having read messages at least weekly, and having sent at least 5 messages in the study (of a year). In a Facebook and smartphone age this is a tiny ‘dose’ compared to many people’s usage patterns.

We need to find ways to rapidly and robustly research the dynamic of open source peer support; combining curated web space such as listservs and efforts to repurpose and colonise digital spaces like Facebook, Tumblr and Twitter for peer support.

The type of sterile, experimental condition created in both of these studies may lead to purity of purpose in terms of methodology, but it places a limitation on the usefulness of the results in the real world.

Blinding was a challenge in both studies in the review. Blinding and randomisation is an ongoing challenge in a field as multiply influenced as mental health. Digital mental health is an exceptionally hard area to create conventional RCT-level evidence in. Ethical considerations around double blinding and randomisation can be a challenge. More obvious is the challenge in creating an experimental model where control can be achieved to the point the intervention at question can be tested. Novel methods of control beyond randomisation are worthy of consideration; managing bias whilst ensuring that those using an intervention are those most able to use it as intended.

In this trial, Kaplan et al recruited only those who hadn’t used online peer support for a year (and might therefore be less familiar with norms and communities) and those who were prepared even in the control group not to use other online peer support for the duration of the study. In 2016, arguably unlike 2011 it is arguable that the bar on using other online peer support is both unethical and in practice impossible since many people use multiple channels simultaneously within and without the health space.

Implications

The evidence we need currently isn’t there, which is a challenge with digital where demand for services and interventions to acknowledge and understand digital lives is pressing.

There are questions as to how the evidence needed will be developed. There a school of thought that says we shouldn’t progress until we have the evidence. In the meantime technology changes at a rapid pace, and digital lives become ever more significant both as determinants of mental health and distress, and as both risk and protective factors in the management of mental health problems. We can’t stand idle, but neither can we jump into the unknown and risk causing harm and wasting public money in challenging times.

We should look to initiatives such as NIHR Mindtech, and the new INTERREG North West Europe Project eMen for innovation in methodologies for evidencing impact.

Just as in the wider mental health environments, we need to be concerned with the ecosystem that creates digital mental capital, and not just on creating clinical interventions and environments for digitising services; either because that achieves the personal outcomes people seek, or more likely because digital delivery is cheaper.

How can we evidence the impact of supporting positive digital citizenship, understanding and using digital assets via digital history taking, or mediate positive social connections by enabling people to use existing tools? These are at least as important as the need to:

First do no harm.

Links

Primary paper

Välimäki M, Athanasopoulou C, Lahti M, Adams CE. (2016) Effectiveness of Social Media Interventions for People with Schizophrenia: A Systematic Review and Meta-Analysis. J Med Internet Res 2016;18(4):e92
http://www.jmir.org/2016/4/e92/ DOI: 10.2196/jmir.5385

Other references

Hoffmann TC, Glasziou PP, Boutron I, Milne R, Perera R, Moher D, et al. (2014) Better reporting of interventions: template for intervention description and replication (TIDieR) checklist and guide. BMJ 2014;348:g1687 [FREE Full text]

Kaplan K, Salzer MS, Solomon P, Brusilovskiy E, Cousounis P. (2011) Internet peer support for individuals with psychiatric disabilities: A randomized controlled trial. Soc Sci Med 2011 Jan;72(1):54-62. [doi: 10.1016/j.socscimed.2010.09.037] [PubMed abstract]

McMillan S. (2002) Exploring models of interactivity from multiple research traditions: Users, documents, and systems. In: Handbook of new media: Social shaping and consequences of ICTS. London: Sage; 2002:163-181.

Rotondi AJ, Haas GL, Anderson CM, Newhill CE, Spring MB, Ganguli R, et al. (2005) A Clinical Trial to Test the Feasibility of a Telehealth Psychoeducational Intervention for Persons With Schizophrenia and Their Families: Intervention and 3-Month Findings. Rehabil Psychol 2005 Nov;50(4):325-336 [FREE Full text] [doi: 10.1037/0090-5550.50.4.325] 48.

Rotondi AJ, Anderson CM, Haas GL, Eack SM, Spring MB, Ganguli R, et al. (2010) Web-based psychoeducational intervention for persons with schizophrenia and their supporters: one-year outcomes. Psychiatr Serv 2010 Nov;61(11):1099-1105. [doi: 10.1176/ps.2010.61.11.1099] [PubMed abstract]

Alvarez-Jimenez M, Alcazar-Corcoles MA, González-Blanch C, Bendall S, McGorry PD, Gleeson JF. (2014) Online, social media and mobile technologies for psychosis treatment: a systematic review on novel user-led interventions. Schizophr Res 2014 Jun;156(1):96-106. [doi: 10.1016/j.schres.2014.03.021] [PubMed abstract]

Highton-Williamson E, Priebe S, Giacco D. (2015) Online social networking in people with psychosis: A systematic review. Int J Soc Psychiatry 2015 Feb;61(1):92-101. [doi: 10.1177/0020764014556392] [PubMed abstract]

Photo credits

• Jason Howie CC BY 2.0
• Pabak Sarkar CC BY 2.0
• Sheltieboy CC BY 2.0
• Kay Kim CC BY 2.0
• mkhmarketing CC BY 2.0