No proof that 85% of mental health apps accredited by the NHS actually work

shutterstock_137348504

Depression Apps: in theory

It’s an inconvenient reality that while demand for psychological services seems to be forever increasing, NHS resources designated for the treatment of mental health problems are going the other way. The unfortunate, but inevitable result is that unmet need for NHS mental health services has reached an unprecedented level (Cooper, 2014).

To illustrate the situation, monthly referrals to community mental health teams increased over 13% in 2013, and 16% in the case of crisis services (The Mental Health Policy Group, 2015), yet roughly 200 full-time NHS mental health doctors, and 3,600 nurses were lost over the same period (Cooper, 2014).

This has meant that despite the ‘new NHS standard’ whereby 95% of those with a mental health issue are to be seen within 18 weeks (NHS England, 2015), 1 in 10 are waiting over a year before receiving any form of treatment, with 1 in 2 waiting over 3 months (We Need To Talk Coalition, 2013).

While this is a problem in itself, a seemingly much bigger problem is the reality that 1 in 6 of those on waiting lists for mental-health services are expected to attempt suicide, 4 in 10 are expected to self-harm and 6 in 10 will likely to see their condition deteriorate before having the opportunity to see a mental health professional (Cooper, 2014).

So how should the NHS and its patients respond? With the widespread availability and increased reliance upon smart phones, one increasingly popular suggestion is the use of apps. Our own experiences can tell us that apps are relatively inexpensive and widely available, but probably most importantly, the use of an app by one person won’t prevent another using the same service at the same time. Given the history of long waiting lists for mental health services this is a highly desirable trait for future NHS services, which could result in flexible, user-led healthcare delivery.

MIND have reported that just 50% and 13% of people currently have a choice as to when and where they receive therapy respectively (We Need To Talk Coalition, 2011), and as such, it’s also possible that the wider-spread use of apps could extend the reach of traditional mental-health services to those who, for one reason or another, are not currently able to engage with treatment. Examples may include the teenager who is too anxious or stigmatised to discuss his condition face-to-face, the armed forces serviceman for whom a desire for anonymity is paramount (Murphy & Busuttil, 2014), or the single-mother who struggles to schedule an appointment around her childcare and work commitments.

With 1 in 10 people with a mental health problem waiting over a year for treatment, perhaps apps can help cut the queues?  

With 1 in 10 people with a mental health problem waiting over a year for treatment, perhaps apps can help cut the queues?

Depression Apps: in practice

Regrettably, the reality is that there’s currently a considerable gap between the benefits that apps may provide in theory, and what they are likely to deliver in practice.

Taking a look at the NHS Apps Library, there are currently 27 mental health apps accredited for use by patients, with a total of 14 designated for the treatment/management of the symptoms of depression and anxiety. Upon close inspection, terms such as ‘control stress’, ‘increase wellbeing’, ‘beat depression’ and ‘improve mood’ are frequently listed benefits from downloading, and often purchasing these apps.

Worryingly though, just 4/14 are able to provide any tangible evidence of outcomes, as reported by real-world users, to substantiate their claims, while just 2/14 make use of NHS-validated performance measures including the Generalised Anxiety Disorder 7 (GAD-7) questionnaire, which is routinely used to assess the effectiveness of other NHS-accredited treatments, including counselling and cognitive-behavioural therapy.

As a result, we are currently facing an open question regarding the true-effectiveness of the remaining 85% (12/14) of NHS-accredited mental health apps.

The majority of depression and anxiety apps accredited by the NHS Apps Library have no tangible evidence of outcomes to substantiate their claims.

The majority of depression and anxiety apps accredited by the NHS Apps Library have no tangible evidence of outcomes to substantiate their claims.

Room for concern?

In 2013, a review of mHealth apps found that from 2003-2013 just 32 articles were published regarding depression apps, compared with a total of 1,536 available for download (Martínez-Pérez et al, 2013). While this finding of a high availability, but low underlying evidence-base is concerning, it could arguably be expected from the open and largely unregulated free-markets that are the app stores. In contrast, the apps under consideration here are the beneficiaries of a ‘seal of approval’ from a world-leading healthcare system, and as a result, the expectation is that they are of significantly greater quality.

This is a worrying situation. Most of us would acknowledge that there is a perceived implicit level of quality that comes with accreditation or association with the NHS, with reputation and legitimacy of sources known to be highly correlated with app downloads (Dennison et al, 2013). Considering that 3 in 10 individuals with an untreated mental health issue currently opt to pay for treatments privately (We Need To Talk Coalition, 2011), the recommendation of, purchase and use of apps that are yet to demonstrate any objectively measurable benefits to users, is not only a potential waste of money, but could also potentially have a compounding and devastating effect on levels of anxiety in those with the greatest need and the least access to effective NHS-led mental health services.

The NHS Apps Library has closed after just 2 years.

The NHS Apps Library has closed after just 2 years.

Moving forward

On the large part, the National Health Service provides a regulatory framework that is second to none, with the attention to detail, rigour and emphasis on safety, clinical quality and cost-effectiveness, seeing NICE tokened as the ‘4th hurdle’ to market access. Unfortunately it would appear that this same level of rigour has not been applied to the apps that the NHS has, until now, recommended to patients.

Although the NHS is pushing for a 21st century approach to healthcare, it’s important that this isn’t achieved through a dilution of quality, and fortunately it would appear that the NHS are now taking this subject more seriously. As of October 16th, the NHS Choices Health Apps Library will officially cease to exist, with the National Information Board considering how alternative models for assessing and regulating health apps may be put in place, and ultimately how quality control can be improved.

In the meantime, until such a framework exists, it’s imperative that those considering downloading mental health apps take a moment to weigh up the available evidence, in order to ensure that apps don’t result in more harm than good. Whilst the app store is often slim on technical information, and sifting through medical publications is far from ideal, there are some clear indications of quality to look out for:

  • Firstly, apps supported by a mental health practitioner are on average more than twice as effective as those from non-practitioner led developers (Richards & Richardson, 2012)
  • Secondly, apps with approval from other well-established regulatory bodies including the US Food and Drug Administration (FDA), can act as an intermediary quality control and help separate those apps which offer users hope, and those which offer real proven solutions
  • Thirdly, are apps forthcoming with the information they provide? It’s easy enough to say ‘this app beats depression’ but do they offer any proof to turn this from what is essentially marketing into evidence of clinical effectiveness?

Finally, it’s worth re-enforcing, that not all mental-health apps are created equally, and that some, designed with clinical quality and effectiveness in mind, are providing real solutions and support to their users. One such app, ‘Big White Wall boasts recovery rates of 58%, which contrasts with the 44% exhibited by the NHS’s flagship ‘Increasing Access to Psychological Therapies’ (IAPT) initiative over the same period, demonstrating that if done properly, apps really can improve people’s mental health, at a low cost and from the comfort of their own home.

Links

Primary paper

Leigh S, Flatt S. App-based psychological interventions: friend or foe? Evidence Based Mental Health 2015 doi:10.1136/eb-2015-102203

Other references

Cooper, C. (2014) Thousands attempt suicide while on NHS waiting list for psychological help. Independent, 16 Sep 2014.

A manifesto for better mental health (PDF). The Mental Health Policy Group, 2015.

Guidance to support the introduction of access and waiting time standards for mental health services in 2015/16 (PDF). NHS England, 2015.

We still need to talk A report on access to talking therapies (PDF). We Need To Talk Coalition, 2013.

We need to talk: getting the right therapy at the right time (PDF). We Need To Talk Coalition, 2011.

Murphy D, Busuttil W. (2014) PTSD, stigma and barriers to help-seeking within the UK Armed Forces (PDF). J R Army Med Corps 2014;0:1–5. doi:10.1136/jramc-2014-000344

Generalised Anxiety Disorder Assessment (GAD 7). Patient website, last accessed 12 Oct 2015.

Martínez-Pérez B, de la Torre-Díez I, López-Coronado M. (2013) Mobile Health Applications for the Most Prevalent Conditions by the World Health Organization: Review and Analysis. J Med Internet Res 2013;15(6):e120 DOI: 10.2196/jmir.2600

Dennison L, Morrison L, Conway G, Yardley L. (2013) Opportunities and Challenges for Smartphone Applications in Supporting Health Behavior Change: Qualitative Study. J Med Internet Res 2013;15(4):e86 DOI: 10.2196/jmir.2583

Richards D, Richardson T. (2012) Computer-based psychological treatments for depression: a systematic review and meta-analysis. Clin Psychol Rev. 2012 Jun;32(4):329-42. doi: 10.1016/j.cpr.2012.02.004. Epub 2012 Feb 28. [PubMed abstract]

Share on Facebook Tweet this on Twitter Share on LinkedIn Share on Google+