Depression Apps: in theory
It’s an inconvenient reality that while demand for psychological services seems to be forever increasing, NHS resources designated for the treatment of mental health problems are going the other way. The unfortunate, but inevitable result is that unmet need for NHS mental health services has reached an unprecedented level (Cooper, 2014).
To illustrate the situation, monthly referrals to community mental health teams increased over 13% in 2013, and 16% in the case of crisis services (The Mental Health Policy Group, 2015), yet roughly 200 full-time NHS mental health doctors, and 3,600 nurses were lost over the same period (Cooper, 2014).
This has meant that despite the ‘new NHS standard’ whereby 95% of those with a mental health issue are to be seen within 18 weeks (NHS England, 2015), 1 in 10 are waiting over a year before receiving any form of treatment, with 1 in 2 waiting over 3 months (We Need To Talk Coalition, 2013).
While this is a problem in itself, a seemingly much bigger problem is the reality that 1 in 6 of those on waiting lists for mental-health services are expected to attempt suicide, 4 in 10 are expected to self-harm and 6 in 10 will likely to see their condition deteriorate before having the opportunity to see a mental health professional (Cooper, 2014).
So how should the NHS and its patients respond? With the widespread availability and increased reliance upon smart phones, one increasingly popular suggestion is the use of apps. Our own experiences can tell us that apps are relatively inexpensive and widely available, but probably most importantly, the use of an app by one person won’t prevent another using the same service at the same time. Given the history of long waiting lists for mental health services this is a highly desirable trait for future NHS services, which could result in flexible, user-led healthcare delivery.
MIND have reported that just 50% and 13% of people currently have a choice as to when and where they receive therapy respectively (We Need To Talk Coalition, 2011), and as such, it’s also possible that the wider-spread use of apps could extend the reach of traditional mental-health services to those who, for one reason or another, are not currently able to engage with treatment. Examples may include the teenager who is too anxious or stigmatised to discuss his condition face-to-face, the armed forces serviceman for whom a desire for anonymity is paramount (Murphy & Busuttil, 2014), or the single-mother who struggles to schedule an appointment around her childcare and work commitments.
Depression Apps: in practice
Regrettably, the reality is that there’s currently a considerable gap between the benefits that apps may provide in theory, and what they are likely to deliver in practice.
Taking a look at the NHS Apps Library, there are currently 27 mental health apps accredited for use by patients, with a total of 14 designated for the treatment/management of the symptoms of depression and anxiety. Upon close inspection, terms such as ‘control stress’, ‘increase wellbeing’, ‘beat depression’ and ‘improve mood’ are frequently listed benefits from downloading, and often purchasing these apps.
Worryingly though, just 4/14 are able to provide any tangible evidence of outcomes, as reported by real-world users, to substantiate their claims, while just 2/14 make use of NHS-validated performance measures including the Generalised Anxiety Disorder 7 (GAD-7) questionnaire, which is routinely used to assess the effectiveness of other NHS-accredited treatments, including counselling and cognitive-behavioural therapy.
As a result, we are currently facing an open question regarding the true-effectiveness of the remaining 85% (12/14) of NHS-accredited mental health apps.
Room for concern?
In 2013, a review of mHealth apps found that from 2003-2013 just 32 articles were published regarding depression apps, compared with a total of 1,536 available for download (Martínez-Pérez et al, 2013). While this finding of a high availability, but low underlying evidence-base is concerning, it could arguably be expected from the open and largely unregulated free-markets that are the app stores. In contrast, the apps under consideration here are the beneficiaries of a ‘seal of approval’ from a world-leading healthcare system, and as a result, the expectation is that they are of significantly greater quality.
This is a worrying situation. Most of us would acknowledge that there is a perceived implicit level of quality that comes with accreditation or association with the NHS, with reputation and legitimacy of sources known to be highly correlated with app downloads (Dennison et al, 2013). Considering that 3 in 10 individuals with an untreated mental health issue currently opt to pay for treatments privately (We Need To Talk Coalition, 2011), the recommendation of, purchase and use of apps that are yet to demonstrate any objectively measurable benefits to users, is not only a potential waste of money, but could also potentially have a compounding and devastating effect on levels of anxiety in those with the greatest need and the least access to effective NHS-led mental health services.
Moving forward
On the large part, the National Health Service provides a regulatory framework that is second to none, with the attention to detail, rigour and emphasis on safety, clinical quality and cost-effectiveness, seeing NICE tokened as the ‘4th hurdle’ to market access. Unfortunately it would appear that this same level of rigour has not been applied to the apps that the NHS has, until now, recommended to patients.
Although the NHS is pushing for a 21st century approach to healthcare, it’s important that this isn’t achieved through a dilution of quality, and fortunately it would appear that the NHS are now taking this subject more seriously. As of October 16th, the NHS Choices Health Apps Library will officially cease to exist, with the National Information Board considering how alternative models for assessing and regulating health apps may be put in place, and ultimately how quality control can be improved.
In the meantime, until such a framework exists, it’s imperative that those considering downloading mental health apps take a moment to weigh up the available evidence, in order to ensure that apps don’t result in more harm than good. Whilst the app store is often slim on technical information, and sifting through medical publications is far from ideal, there are some clear indications of quality to look out for:
- Firstly, apps supported by a mental health practitioner are on average more than twice as effective as those from non-practitioner led developers (Richards & Richardson, 2012)
- Secondly, apps with approval from other well-established regulatory bodies including the US Food and Drug Administration (FDA), can act as an intermediary quality control and help separate those apps which offer users hope, and those which offer real proven solutions
- Thirdly, are apps forthcoming with the information they provide? It’s easy enough to say ‘this app beats depression’ but do they offer any proof to turn this from what is essentially marketing into evidence of clinical effectiveness?
Finally, it’s worth re-enforcing, that not all mental-health apps are created equally, and that some, designed with clinical quality and effectiveness in mind, are providing real solutions and support to their users. One such app, ‘Big White Wall’ boasts recovery rates of 58%, which contrasts with the 44% exhibited by the NHS’s flagship ‘Increasing Access to Psychological Therapies’ (IAPT) initiative over the same period, demonstrating that if done properly, apps really can improve people’s mental health, at a low cost and from the comfort of their own home.
Links
Primary paper
Leigh S, Flatt S. App-based psychological interventions: friend or foe? Evidence Based Mental Health 2015 doi:10.1136/eb-2015-102203
Other references
Cooper, C. (2014) Thousands attempt suicide while on NHS waiting list for psychological help. Independent, 16 Sep 2014.
A manifesto for better mental health (PDF). The Mental Health Policy Group, 2015.
Guidance to support the introduction of access and waiting time standards for mental health services in 2015/16 (PDF). NHS England, 2015.
We still need to talk A report on access to talking therapies (PDF). We Need To Talk Coalition, 2013.
We need to talk: getting the right therapy at the right time (PDF). We Need To Talk Coalition, 2011.
Murphy D, Busuttil W. (2014) PTSD, stigma and barriers to help-seeking within the UK Armed Forces (PDF). J R Army Med Corps 2014;0:1–5. doi:10.1136/jramc-2014-000344
Generalised Anxiety Disorder Assessment (GAD 7). Patient website, last accessed 12 Oct 2015.
Martínez-Pérez B, de la Torre-Díez I, López-Coronado M. (2013) Mobile Health Applications for the Most Prevalent Conditions by the World Health Organization: Review and Analysis. J Med Internet Res 2013;15(6):e120 DOI: 10.2196/jmir.2600
Richards D, Richardson T. (2012) Computer-based psychological treatments for depression: a systematic review and meta-analysis. Clin Psychol Rev. 2012 Jun;32(4):329-42. doi: 10.1016/j.cpr.2012.02.004. Epub 2012 Feb 28. [PubMed abstract]
No guaranty of ‘appiness, suggests piece from @Mental_Elf https://t.co/1NNdebPNYp
@Dr_JB_Kirkbride @Mental_Elf Interesting. Because many people have become heroes in retailing their inclusive development.
No proof that 85% of mental health apps accredited by the NHS actually work http://t.co/QFoWPYi22s #MentalHealth http://t.co/itLusXLGxV
Be wary of claims made by mental health apps: http://t.co/m5669RzVCt
App-based psychologcal interventions: friend or foe? Free @EBMentalHealth http://t.co/7GRxtwPKXq + blog @Mental_Elf http://t.co/7gHAWeuMgU
@Mental_Elf levels of digital participation & more potential to increase health inequality?? Many considerations…
@Mental_Elf Great blog today! But worrying news…
Thank you Simon for this great blog summarising an important area. It’s certainly true that this message needs to reach the people who are downloading and using apps, which they no doubt presume are reliable. I hope that our readers investigate this issue further and read the free full-text article available on the Evidence-Based Mental Health Journal website: http://ebmh.bmj.com/content/early/2015/09/16/eb-2015-102203.full
You mention the demise of the NHS Apps Library, which was recently announced. Within mental health, the apps library has been replaced by a page on the NHS Choices website which brings together a number of leading online mental health services (e.g. Big White Wall, Buddy, FearFighter, Ieso, Kooth, Silvercloud and Sleepio). See more here: http://www.nhs.uk/conditions/online-mental-health-services/pages/introduction.aspx
I’m not sure how many of these apps use validated performance measures and have high quality research supporting their safety and effectiveness, but it seems to me that this list has more of a feeling of robust and reliable digital services than the 27 mental health apps previously accredited and included in the old NHS Apps Library. What do you think?
It will be interesting to see how the NHS take this forward. I note that Alexia Tonnel from NICE is giving a keynote speech at the MindTech conference in December on the National information Board’s plans to evaluate digital mental health products and make accredited products available via NHS Choices: http://www.mindtech.org.uk/mindtech-annual-conference.html
There are a couple of consultation documents on this for readers who want to find out more:
https://www.nationalelfservice.net/cms/wp-content/uploads/2015/10/NIB1.2-slides-June-2015_for-consultation.pdf
https://www.nationalelfservice.net/cms/wp-content/uploads/2015/10/Workstream-1.2-roadmap-long-version-June-2015.pdf
Cheers,
André
Hi André, I share your enthusiasm for engaging with potential users of such apps. While a great number of apps are proving to be very effective (Big White wall and Leso digital health have demonstrated recovery rates far in excess of those achieved by IAPT), the are just as many, if not more, that have little if any credible evidence underpinning them.
I would agree that the new approach to offering mental health apps through NHS choices appears more rigourous, and the apps included, more effective than those included in the NHS apps library. However, it would still of great benefit if a structure was formalised sooner rather than later, much like with NICE’s health technology assessment of pharmaceuticals and other health generating products, such that developers can have clear indications as to what constitutes a ‘good, scientifically credible, well-validated app’, and also understand what is classed as an acceptable evidence of effectiveness.
Thanks,
Simon
No proof that 85% of mental health apps accredited by the NHS actually work – http://t.co/jTv6UblRvN #mentalhealth
Be wary of claims made by mental health apps http://t.co/cDkZhyTrWo
RT @Mental_Elf: 1 in 10 people with a #MentalHealth problem wait over a year for treatment. Perhaps apps can help cut the queues? http://t.…
“No proof that 85% of mental health apps accredited by the NHS actually work” http://t.co/pOjIBRYGBk
but…some proof that 15% of mental health apps accredited by the NHS DO actually work? http://t.co/pOjIBRYGBk
No proof that 85% of mental health apps accredited by the NHS actually work https://t.co/vXb6GQ2g1f via @sharethis
No proof most NHS accredited mental health apps actually work http://t.co/O3eQRJBSW0 via @rightrelevance thanks @mental_elf
@LouMatter @forestimaginatn @IsmaelVelasco @towndigitalhub http://t.co/wyvrlRuQNw
“…it’s imperative that those considering downloading mental health apps take a moment to weigh up the available evidence, in order to ensure that apps don’t result in more harm than good.”
It may be imperative, but, from my own experiences, this is not a population likely to be able to do so.
It’s hard enough simply trying to get information on local mental health services online, nevermind wading through a mire of apps all promising benefits and evaluating there respective positives, negatives and evidence bases.
So how should we support the general public to become more discerning and savvy about app quality? Or are you saying that this is just impossible from the word go?
How do people assess quality right now? Simon’s point is about the trust we all put in the NHS brand, which when attached to a collection of apps, heavily implies that these apps are safe and effective. Unfortunately this is not the case.
I think the general public are a lot more discerning about health websites than they were 10 years ago. We have all learnt from initiatives like DISCERN, even if these are skills we’ve picked up surreptitiously along the way: http://www.discern.org.uk/discern_instrument.php
I guess the world of apps is still so young that we are still searching for the best ways to critically analyse them. For me, the core principles are the same. We need to be assessing accessibility, usability and reliability.
Cheers, André
My comment is only that it is un-realistic to believe that someone in the throws of what could be their first episode of depression is well suited, or able, to evaluate (on their own) the relative evidence-based merits of different options for apps to self-manage their condition.
If you’ve barely got the energy or willpower to take a shower, or get dressed, are you really going to have the energy to research multiple apps?
If you already have huge doubts about your own ability to do things, are you going to trust in your own ability to ‘choose the right app’ among the hundreds claiming effectiveness, when you’re feeling completely hopeless anyway?
I would argue that it’s pretty unlikely not.
A rubber stamp of approval from something like the NHS, or MIND, makes these choices far easier. Sources you’re likely to be looking through, or for, anyway.
Which is why I whole-heartedly agree with Simon’s point about trust in the NHS and people’s assessment of the value of an app they advise. Trust which, depending on the app, may be misplaced.
My feeling is that this is an ideal area in which the NHS, through the NIHR, should be investing in. They should be producing an app and evaluating it as rigorously as they are able. In partnership with someone, preferably.
I think there is enormous, positive scope for such an app to help those with depression, or other mental health issues.
I would agree that the general public are far better at assessing these kinds of things than they were ten years ago, but that doesn’t necessarily apply to someone with depression. It may not be that they are able; they might just not care.
Most NHS accredited depression & anxiety apps have no tangible evidence of outcomes to substantiate their claims http://t.co/WCZgEQBbXq
@Mental_Elf Curious, what is their approval process & surely such apps are medical devices which have strict criteria? Who defines if meddev
RT @meta4RN: Mental health problem?
There’s an app for that… but it is it any good? http://t.co/6a9gQgIIHh #mHealth HT @Mental_Elf
RT @Mental_Elf: What makes a good, scientifically credible, well-validated health app? http://t.co/WCZgEQBbXq http://t.co/9Q3PEaMoTK
La mayoría de las apps para depresión y ansiedad acreditadas por el NHS no tienen ninguna evidencia de su utilidad http://t.co/Tzf2LtB5va
This is a useful article.
I’ve contended for a while that the NHS is in the mire when it comes to assessing the affective elements of apps and online services. It doesn’t really know what it’s look for, how to look for it or even how to understand it if it finds it.
There’s an attitude of refusing to meet the developer world half way; almost looking forward to the opportunity to say ‘you’ve got that wrong’ tutting and shaking its collective professional head while failing to acknowledge that it has done nothing to help anyone do anything ‘right’. At present the interface between mental health and tech development is poor at best. Clinicians consistently undervalue the profession of app development and user-centred design, assuming that ‘anyone can do it’ whilst often failing to put into the public domain useful knowledge that that would enable developers to bring their skills to bear upon problems and challenges in mental health in an evidence informed way.
It’s correct that many mental health apps are crap. But much of just about anything is crap. Judgement of some individual examples should not be used as a ruling on the potential of a field.
At present there is a wall between the people doing the tech thinking (who will often tend to work in for profit business) and those who ‘know’ about mental health who will tend to work within the NHS, charity sector or academic institutions. Scepticism about the benefits of technology leads to a stunted, awkward interaction between those sides where the common ground cannot be established to develop useful, measurable apps and services.
There is often a refusal on the public sector side this relationship to discuss business and revenue models for apps, and a refusal to discuss the financial challenges of trialling and iteratively developing mental health apps and services. At present, often the only way of robustly trialling mhealth apps is to find a university department that a) has funding and b) is already thinking a long the same lines.
The problem or challenge is that the tech possibility runs far quicker than the mental health world’s ability to get it’s head around it and translate that into research and funding decisions.
As the holders of both the purse strings, and the knowledge assets for mental health, the NHS should become an investor of first choice if it is serious about getting the mental health apps and digital services that it appears to want. It can bring the knowledge assets and the framework for interacting with possible trial subjects and with existing intelligence and can smooth path for people to carry out proper user-centred app design by working with people who actually experience particular conditions and illnesses.
At the moment, rather than being a true partner The NHS is more like an in-law, tutting and shaking its head that tech is doing it wrong but giving no guidance as to how mental health apps might do it right.
@markoneinfour (dashed this off on the train, may return to it later)
Great post Mark! :-)
I completely agree with your point about more joined up working. The best apps come from true collaboration with a mix of service users, professionals, developers, researchers and funders.
So who is developing apps in this way? Let’s hear some inspirational stories.
Cheers,
André
RT @Mental_Elf: Are you developing mental health apps in a truly collaborative way? Share your experiences on our blog >>> http://t.co/WCZg…
This is a very interesting article and I think it is a good thing that light is finally being shone on the shambolic NHS health apps library which I have been involved with since it’s launch several years ago.
I was involved with a team that was one of the first apps on the library, in fact we were one of the launch apps. The on boarding process and validation of apps was awful with very little if any that I could see clinical input. It was all very rushed in time for the NHS Innovation Expo this year and at the time I was surprised at the lack of clinical evidence needed to get onto the library. I do know that we had to answer a lot of questions and do a lot of paperwork but since the launch it seems more and more apps have been added with less and less information so I wonder what process they had to go through to get onto the library and how rigorous it was.
Since then I have heard the library referred to the “NHS’ best kept secret” because any news of publicity of it has been absolutely buried and the NHS stopped talking about it at all about a year after launch. Since then we have had talk of accreditation and kite marking but this seems to change on a monthly basis making it very difficult for both consumers (to know what apps are safe and effective) and developers (to know what hoops they will have to jump through).
I’m also curious as to what the article means by clinically validated as many of the tools on the library are not clinical interventions and so do not necessarily need to go through the same rigorous testing. Also what standard of clinical evidence was required for apps to pass the study’s test? If it was an RCT then that potentially makes the statistics in this article very misleading as many innovations and tech products are not suitable for this kind of evaluation – something that has been argued for a long time now.
As Mark has already pointed out, one of the main issues is that the majority of people involved in this kind of work from the NHS’ side of things do not know what they are looking for
Hi,
I found your comment around seemingly unending goalpost shifting interesting. I imagine a lot of this was ultimately down to the NHS jumping in at the deep end and then ‘learning’ as they progressed with the apps library, unfortunately though it is still unclear exactly how apps were appraised before acceptance into the library.
To answer your question, the methodology of the analysis was simple, and we tried to keep things as broad and inclusive as possible. We looked at what apps claimed they did/could do, on the NHS apps library, both android, and apple app stores and any websites dedicated to these apps. We then simply looked for ‘any’ evidence to corroborate these statements, whether a report that stated 100 people used the app and 50 had an X% reduction in the symptoms of depression, or much more rigorous in terms of an RCT or an observational study.
We then looked (for any study where evidence was provided) at the quality of this evidence in comparison to the quality of evidence we may expect if evaluating other NHS approved activities, including antidepressants, counselling etc.
Our criteria for NHS approved/generalisable metrics were that evidence must be in the form of a generalised anxiety disorder -7 question scale (GAD-7), WEMWBS or PHQ-9, as these are commonly used within the NHS and are the metrics of choice for comparator and substitute therapies provided by the NHS’ flagship Increasing Access to Psychological therapies (IAPT) initiative.
I hope this helps clarify things.
Kind regards,
Simon
RT @Mental_Elf: Don’t miss: No proof that 85% of mental health apps accredited by the NHS actually work http://t.co/WCZgEQBbXq #EBP
I use apps not ones the NHS have on their list but ones I have tried myself or my peers have recommended . I take them or leave them as I please. Honestly it doesn’t always take a professional to tell me what’s good for me I can make that decision. Apps are out there people are using them and meanwhile the NHS trudges miles behind … We are just experimenting ourselves I like it better that way .
RT @SalfordRD: An interesting read, showing need for more research into health apps | #askforevidence | http://t.co/VhRA862CAv https://t.co…
No proof most NHS accredited mental health apps actually work http://t.co/aHv4EtCE8e #Apps #MentalHealth #EvidenceBase #Depression
There’s a recent NICE guideline/review on effectiveness of digital in adolescents that shows a positive effect. Difficulty is that for some it really works and you get a good result (the analysis from BWW shows this I think) but for others it’s really not effective. This means average effect size in a pop can either look modest or impressive depending on how you cut the data. Suspect that trick is working out what works for whom.
Issue regarding validity/efficacy a challenge. Tension between developing innovation in timescale that enables a startup to launch app and test, and getting enough users so that evaluation is possible and things can be iteratively improved. On one hand we really need innovation in NHS on other we prefer for RCTs before we decide it’s good enough.
I’ve no idea what’s right, but it seems to me that until we create environment that enables innovation we won’t ever get to point where we have great solutions that help with some of our current challenges.
Hi Anna, I think these are all very good points around the issue, thank you for getting involved.
Regarding average treatment effect, it may be that we are able to identify which sub groups of individuals, whether by age, or specific type of mental illness etc. are more conducive to successful outcomes using apps. One possibility regarding the selective reporting of results is a kind of clinical audit prior to NHS approval. If apps are used and downloaded through the android/apple app store and not the NHS health apps library, there is potential that data reported could be contrasted against sales data to gain an estimate of what proportion of total available results are currently being reported. Alternatively if these apps are obtained via other means this obviously becomes much more difficult.
R.e. the desire for RCTs and strong credible evidence, there is that clear trade off between getting things to market for the potential gains to users right now, and the alternative of being certain before hand that these apps are effective before widespread use (or that they at least ‘first do no harm’). The European medicines agency (EMA) have proposed ways around similar problems for promising and novel, yet currently un-assessed/un-regulated pharmaceuticals. Their requirements are along the lines of, if market access is granted early, all evidence resulting from use within the healthcare system is transparent and ready for analysis by regulators in the form of real-world evidence (RWE), perhaps a similar system would work here. NICE are now increasingly accepting RWE as a valid form of evidence due to increased validity compared to often cherry picked RCT participants, and are becoming less reliant on the gold standard RCT. I also believe the cancer drugs fund will soon trial/are in the process of trialling a similar initiative, so there are some possibilities there for a similar initiative.
I think one of the most important first steps from the NHS would be to openly announce what is acceptable and what isn’t and provide some form of framework such that developers know what is expected from the start. This way we can design the development of apps more efficiently and give clear guidance to developers as to which apps (and what level of underlying evidence) are likely to receive accreditation by the NHS. I suppose they’re currently not sure exactly what it is the NHS want given the variable standards (in terms of evidence generation and design) shown in those accredited so far.
Kind regards,
Simon
I think for developers the most important thing is being able to cost the process into the overall cost of development. Guidance will help to clarify this greatly.
It’s exceptionally important that NHS can illustrate at least guide price for the process of evaluation and testing. The only way that investment is really viable is if costs like this can be confidently quantified.
It will also, perhaps, bring a sensible discussion to the fore about where the investment for such services and apps might come from and what, if any, the value of the potential market actually is.
I don’t say this from a cut throat business point of view; I say this from a good five or six years experience of not being able to give very definitive advice about the business models for delivering mental health relevant services in a sustainable way.
@markoneinfour
No proof most NHS accredited mental health apps actually work http://t.co/yq0SKSmkY3
No proof that 85% of mental health apps accredited by the NHS actually work https://t.co/HSlivpxYjM
No proof most NHS accredited mental health apps actually work http://t.co/yq0SKSmkY3 via @Instapaper
NHS Health apps library currently closed – data privacy issues, no proof MH apps work – @Mental_Elf takes a look http://t.co/iiRNqImNt1
The mention of Happtique in the primary paper is interesting considering what happened to them has also happened to the NHS Apps Library – a publication reveals security flaws in the apps they endorse and they slowly bow out of existence (http://mobihealthnews.com/28165/happtique-suspends-mobile-health-app-certification-program/ and http://www.ukauthority.com/news/5698/nhs-england-closes-health-apps-library). A potential user has far more of an issue with whether an app is going to leak their health data (which by the way we would pay $59.46 to protect https://hbr.org/2015/05/customer-data-designing-for-transparency-and-trust?utm_campaign=Socialflow&utm_source=Socialflow&utm_medium=Tweet) than whether it has an evidence-base behind it.
The reality is that we find it easy to download, critique and delete an app (although dropping out of an IAPT appointment appears to be nearly as easy!). As Leigh mentions in the paper, engagement can be incredibly important when it comes to achieving outcomes and technology has many tricks up its sleeve. Others have already pointed out the impossible dilemma that faces all stakeholders in this field of trying to provide evidence or regulation at the rate of innovation and uptake. For developers therein lies the issue of financing, for researchers the issue of development, and for the NHS the issue of both.
I can see many ways in which technology helps in the area of mental health, not just by providing an intervention but also providing an adjunct to therapy, a pathway to an appropriate service, and a way to improve mental health literacy. This means accepting outcomes other than validated metrics, although they of course play a role. In a population that goes online before going to a doctor (http://www.pharmafile.com/news/497975/uk-patients-self-diagnose-rather-visit-gp), must wait for months to access psychological therapies, and who often wants to just deal with it themselves (http://www.ncbi.nlm.nih.gov/pubmed/23931656) I believe there is a role for apps other than those delivering evidence-based interventions.
But can we prevent harm? I believe the greatest harm comes from a lack of security and safety. The nocebo effect is used to describe when therapy or a supposedly harmless placebo has a harmful effect. A mental health app has the potential to cause harm through a failure to consider the unintended consequences and through actively causing a negative thought or behaviour to be reinforced. It might also cause someone to consider it an alternative and not find appropriate help. How can this even be detected? In the real world all our regulations and research often fails to account for actual usage.
Some attempts have been made to find a solution for health apps in general – the British Standards Institute (http://shop.bsigroup.com/forms/PASs/PAS-2772015/), the Digital Health and Care Alliance, the European Commission (https://ec.europa.eu/digital-agenda/en/mhealth), the MHRA – but most are faced with exactly the same problem that the FDA and NHS face. It is reliant on the developer’s intended use. The use of words like ‘anxious’ and ‘stress’ to describe our general day to day wellbeing places mental health as a concept far from the concept of mental illness (this was also discussed in CMO Annual Report 2013) and opens up apps to an avoidance of regulation, even within a structure that purports to cover it.
Does this mean that all NHS accreditation of mental health apps is doomed to fail? No, but it requires a different approach. Rather than focusing on finding evidence for an apps efficacy for a specific condition (and thus placing us into another decade of quackery and avoidance of responsibility before research can catch up) it should be based on the need it can provide for. Those outside of mental health services who need encouragement to seek help should have apps designed to provide this and their value appropriately weighted – many apps with moderated forums already do this. Support on the days without therapist contact. An app that can help them keep on top of the CBT skills they learn. Somewhere they can track their physical and mental health. Evidence where evidence is due – if an app delivers an intervention it should be evidence-based but there are already studies out there that have proven how effective certain functions are (for instance self-monitoring in http://www.biomedcentral.com/1471-244X/13/312). It is also important that their users, possibly measured through reviews or usage analytics, accept them. But most important is that they are intentionally aimed at a relevant and appropriate population as a way to help avoid any possible nocebo effects.
http://t.co/lshH5WoHDi @VictoriaBetton
Very interesting article (and others’ comments are possibly even more interesting). I’m an app developer and thought I would drop in my two penneth worth…
A ‘traditional’ app developer view might be something along the lines of “if you want better apps you need innovation, and to get innovation you need many apps, many attempts and many failures (ideally with the failures occurring rapidly) so their demise leads to a better next generation of apps – and if you try to regulate too much you will strangle innovation…”
I think I probably agree with these “let a thousand flowers bloom” and “let the cream rise to the top” approaches – BUT not at the cost of putting out apps that do harm…
So how do we improve quality of the apps? The usual regulatory approach would be to test the app against a series of standards – probably at the end, or close to the end of the development process – and it sounds like this will continue to be needed in some form.
How about at the other end of the development process where people are dreaming dreams and putting together app development projects? It is not uncommon for me to be involved in (non-health related) app development projects where some of the basics haven’t been thought through – which usually means the resulting app didn’t end up as good as it might otherwise have.
I bet that if a health-related app development project has input from clinicians, users and developers, has developed other partnerships as needed, has spent time on the reasons for the app, is clear on what the app should do, has sorted out metrics for success, AND processes for testing the app – then it’s much more likely to create a better app.
I’m slightly involved in a thing called Code4Health (https://code-4-health.org) as a sub-contractor running introductory coding workshops around the country. I really like the things they are doing around building communities of innovation and should think what they do will be of interest to others here.
@TcameronTodd @DrBastianSeidel Mental health apps have a woeful evidence-base https://t.co/F30waFzOVl via @Mental_Elf
@MarkOneInFour cites our recent blog https://t.co/QhrKDb5UzL #mindtech2015
[…] in a discussion at the Mental Elf site about the lack of clinical evidence for many popular depression apps I said: “At the moment, rather than being a true partner The NHS is more like an in-law, tutting […]
[…] a recent Mental Elf blog by health economist Simon Leigh (who is here today) showed that 27 mental health apps were […]
[…] http://www.nationalelfservice.net/mental-health/depression/no-proof-that-85-of-mental-health-apps-ac… […]
[…] 2013). Even with respect to NHS-accredited app-based psychological interventions, historically, as few as 15% have been backed by data to corroborate claims of effectiveness (Leigh et al, […]
[…] No proof that 85% of mental health apps accredited by the NHS actually work […]