A new ‘Insights’ piece has just been published in World Psychiatry (the Official Journal of the World Psychiatric Association) by John Torous et al, in which they seek to propose a consensus for a set of standards and principles for the evaluation of mental health apps (Torous et al, 2019).

So why is this consensus piece so timely? As the authors note, whilst there are over 10,000 mental health apps available to download, there are few resources available to help users (people accessing mental health services, practitioners and organisations) evaluate the quality and suitability of these tools and services. It is currently not a straightforward task for users to appraise mental health apps; and how can practitioners confidently recommend digital mental health tools to the people they support, without clarity on minimum standards? It is comparatively easy to design and develop a mobile app, hence their proliferation in the marketplace, but creating a tool that is safe and efficacious is a more significant challenge.

Recommendations

Through deliberation with experts in the field, the authors have identified four key topic areas where they argue minimum standards should be articulated and met. Their recommendations are summarised below:

1. Data safety and privacy

• Data storage, use and sharing practices should fulfil healthcare standards for handling patient health information data
• Standards must be transparent to the user
• The end user must have the option to “opt out” of sharing their information
• Any language regarding data storage, use and sharing must be written at a maximum of a 6th grade (year 7 in England) reading level
• Technical security reviews and data audits are necessary to guarantee that apps follow the standards they set out and ensure that new vulnerabilities are quickly identified.

2. Effectiveness

• Clinical trials to determine the efficacy and effectiveness of apps with re-evaluation based on changes to the app
• A nosology (classification scheme used in medicine to classify disease) for mental health apps and guidelines to match the necessary level of evidence for each app’s use cases and risks.

3. User experience/adherence

• User-centred/user experience (UX) design methods should be employed when creating an app
• This includes involving the intended end-user in the development, and when usability is evaluated, developers should report use statistics to all end users
• Standards concerning best practice in user design research for mental health apps should be articulated.

4. Data integration

• Mental health apps that are intended to be used in conjunction with health care systems should employ methods to ensure interoperability with electronic health records
• Mental health apps will need to document the processes they use to ensure the secure exchange of information between platforms
• Internal data standards for interoperability are needed.

The authors note that US and UK governments and associated organisations are progressing work in respect of the above four thematic areas. They conclude by recommending that this work is brought together so that a universal set of standards for mental health apps can be agreed (Torous et al, 2019).

So how are we faring in England?

There has been significant investment from NHS England and others over the last few years to create the conditions for health apps to be designed and developed so they can be trusted by end users. This is a world away from when we first started out as mHabitat in 2014, where we regularly encountered any number of ill-thought-through apps springing from a well intentioned practitioner and their mate developer who were in blissful ignorance of the hurdles they would encounter. In my imagination, there is a very large app graveyard somewhere in the cloud where these ill-conceived apps are ignominiously buried.

Whilst work led by the NHS is still in its infancy, notable developments over the last couple of years are:

• Code of conduct for data-driven health and care technology (Department of Health & Social Care; 2018)
  • A set of principles that describe what the NHS expects from suppliers of data-driven technologies, and a set of commitments that outline what the government will do to support and encourage innovators in health and care.
  • The aim of the code is to make it easier for innovators to understand what the NHS needs from them, and to help health and care providers choose safe, effective, secure technology to improve the services they provide.

• NHS Digital, data and technology standards framework (NHS Digital; 2018)
  • Sets out expectations around the use of data, interoperability, and design standards within the NHS.

• Evidence standards framework for digital health technologies (National Institute for Clinical Excellence; 2018)
  • The aim of these standards is to make it easier for innovators and commissioners to understand what good levels of evidence for digital healthcare technologies look like, while meeting the needs of the health and care system, patients, and users.

These standards and code of conduct are complemented by a process for health apps to be reviewed and assessed for inclusion in the public-facing NHS Apps Library (launched April 2017). The library includes a site for developers; Orcha and Our Mobile Health undertake assessments of apps on behalf of NHS Digital before they are approved for inclusion in the library.

A confusing space for developers

All of the above are currently in ‘beta’ mode which illustrates the extent to which standards, frameworks and sites are still being tested and developed as the field of digital health matures. It remains a confusing space for developers, in which finding the right resources is not a straightforward task. There is work to be done to simplify the standards landscape at a national level, whilst also being mindful of the authors’ challenge to join up the international picture (Torous et al, 2019).

It would be remiss of me to not mention the recently published NHS Long Term Plan, which has a whole chapter dedicated to digital technology and a vision to offer a ‘digital first’ option to most of us. The push towards digital in the NHS looks only to gather pace over the coming years.

Designing digital tools that fit into existing services

I’d like to conclude this blog by focusing on the recommendation within theme 4 of the World Psychiatry paper (Torous et al, 2019) that workflow analysis is undertaken to ensure mobile apps are useful and usable and fit into the fabric of the user’s life without creating unnecessary burden. It is our experience at mHabitat that this critical aspect of design is often overlooked. Failing to consider how a digital tool impacts on the clinical workflow can make the difference between adoption and rejection. Without (re)designing services enabled by digital (rather than designing digital tools that then need to fit into a service) it is almost inevitable that extra work will be created.

It is often the case that developers underestimate the complexity of health systems and that practitioners are naive and daunted by the digital sector. However, collaboration between developers and practitioners, along with co-design with patients and users, is vital if we are to create digital tools that make a positive difference to the lives of people accessing and working in mental health services.

Conflicts of interest

Victoria is founder and managing director of mHabitat, an NHS hosted team who help people accessing and working in health and care services, industry and academia collaborate for digital innovation.

Links

Primary paper

Torous J, Andersson G, Bertagnoli A, Christensen H, Cuijpers P, Firth J, Haim A, Hsin H, Hollis C, Lewis S, Mohr DC, Pratap A, Roux S, Sherrill J, Arean PA. (2019) Towards a consensus around standards for smartphone apps and digital mental health. World Psychiatry, 18: 97-98. doi:10.1002/wps.20592

Other references

Department of Health and Social Care (2018) Code of conduct for data-driven health and care technology. Published 5 September 2018.

NHS Digital (2018) BETA – NHS digital, data and technology standards framework.

NICE (2018) Evidence standards framework for digital health technologies.  National Institute for Clinical Excellence.

Photo credits

• Photo by Charles Deluvio ???? on Unsplash
• Photo by Kristopher Roller on Unsplash
• Photo by Igor Starkov on Unsplash
• Photo by Bernard Hermant on Unsplash
• Photo by rawpixel on Unsplash