Arts therapies have been used for a long time to help people with mental health difficulties, and NICE even recommends that they should be offered to everyone with a diagnosis of schizophrenia and related problems such as schizoaffective disorders and psychosis. However, the evidence base even for this group is comparatively weak, and for other mental health problems it is even more so. There is evidence that it might help, but studies are generally small and/or low quality so that drawing firm conclusions is difficult to impossible.
On a personal note, this elf, while never having engaged in music therapy in a clinical sense, has been a musician for a long time and finds making music extremely therapeutic. So this area of research is of interest to me. However, I also have experience of the fact that certain kinds of music-making, such as free improvisation, are not everyone’s cup of tea.
This paper, published in the Journal of Child Psychology and Psychiatry in 2017, was the first to examine the use of music therapy in young people with mental health problems on a large scale.
Free improv: not everyone’s cup of tea, but some of the more radical elves are very partial to a bit of Derek Bailey.
Methods
This was a multi-centre, randomised controlled trial which sought to ascertain the effect of music therapy on children and adolescents between 8 and 16 years with common mental health issues. Participants were recruited from six community care CAMHS sites within the Belfast Health and Social Care Trust in Northern Ireland. Participants had to have a working diagnosis of a mental and behavioural disorder as defined in the ICD-10 (codes F00-F99). In the demographic information, the authors tell us that 53% of participants had a diagnosis that included a form of depression, 37% had a diagnosis that included a form of anxiety, and 19% had a diagnosis that included a form of autistic disorder. As this adds up to 109%, these numbers show that some of the children had more than one of these diagnoses.
Children and adolescents were excluded if they:
- had chronic and severe substance abuse
- had psychosis
- had displayed repeated suicidal behaviour
- were unable to complete self-administered questionnaires with assistance
- were unable to attend music therapy sessions
251 young people in total were randomised to two groups:
- The control group received usual care, which the paper defines as medication and/or psychiatric counselling
- The intervention group received usual care plus music therapy. One therapist delivered all the music therapy sessions. Sessions took place weekly over a period of 12 weeks and lasted 30 minutes. They were based on a model of free improvisation, which is thought to be particularly useful for those who struggle with communication and interpersonal skills.
The primary outcome was communicative and interactional skills as measured by the Social Skills Improvement System rating scales at week 13. This is a self-report questionnaire aimed at both children and their guardians. The child completes one copy and the adult another.
Secondary outcomes included communicative and interactional skills at week 26, self-esteem and depressive symptoms, social functioning and family functioning.
Neither parents nor patients were blind to study condition. Taken together with the fact that outcomes were based on self-report, this represents a serious limitation. Adding an outcome that was not based on self-report such as observation by a clinician, or validated instruments that were more diagnosis-specific, would have helped.
Participants represented a typical CAMHS population but excluded the most severe cases. They were matched in terms of diagnostic group and age. They were recruited from six different sites that varied in terms of socioeconomic background of their clientele.
It is debatable whether the groups were treated equally apart from the experimental intervention. The authors state in their conclusion that some of the observed effect was likely due to the care and attention from the therapist rather than specific to music therapy, but they then go on to say that the inclusion of a non-musical control would have been inappropriate as therapeutic interaction is an active component of music therapy. I would like to question this, as I am of the opinion that attention control could have been achieved.
Results
Participants were analysed in the groups into which they had been randomised (intention-to-treat analysis). There was a lot of missing data: 47 of 123 participants of the intervention group and 23 of 128 participants of the control group were lost to follow-up at week 13. Looking at the table for the primary outcome in the overall population, no significant differences were found between intervention and control groups. The authors performed several subgroup analyses by gender, diagnosis, and age, and found a small but significant improvement in SSIS in participants over 13. However, this was not sustained in the longer term.
The main limitation of the study is its large attrition rate and loss to follow-up. 38% of the intervention group dropped out as compared to 18% of the control group. This seems especially frustrating as treatment acceptance seemed to be quite high: 12% of the intervention group did not attend their first appointment, but 88% of the group (108 young people) attended 10-12 music therapy sessions with all of these attending the final session. However, only 76 young people attended primary outcome data collection sessions.
32 of the young people in the intervention group who had attended 10-12 music therapy sessions (that’s nearly a third of this very adherent group) did not attend primary outcome data collection sessions. The authors speculate about reasons for this and mention substantial parental anecdotes about improvements that may have resulted in parents not wishing to subject their children to further questionnaires about mental health. The amount of missing data may have resulted in an underestimation of the treatment effect.
The large attrition rate and loss to follow-up are a serious limitation of this study.
Conclusions
It’s vitally important to build an evidence base for creative therapies, but the large attrition and loss-to-follow-up rate means that this study’s results probably aren’t very valuable.
The subgroup analyses are interesting, but as with all subgroup analyses (Oxman et al, 1992), findings may be artifactual rather than represent solid evidence.
The authors mention that there was anecdotal evidence that it really worked for some of the children, but this is not captured in the quantitative results. It is also clear that the therapy was completely unacceptable to others. I think it would be useful to study this in a more focussed way. For example, one could include some “harder” outcomes, narrow down the age range and/or diagnosis. This would help to get a clearer picture of the kind of population that is likely to benefit from music therapy.
Once again, a randomised controlled trial fails to find much benefit from music therapy. Where does this leave patients and policy makers?
Links
Primary paper
Porter S, McConnell T, McLaughlin K, Lynn F, Cardwell C, Braiden HJ, Boylan J, Holmes V. and the Music in Mind Study Group (2017) Music therapy for children and adolescents with behavioural and emotional problems: a randomised controlled trial. J Child Psychol Psychiatr, 58: 586–594. doi:10.1111/jcpp.12656
Other references
Oxman AD, Guyatt GH. (1992) A Consumer’s Guide to Subgroup Analyses. Ann Intern Med. 1992;116:78-84. doi: 10.7326/0003-4819-116-1-78
Photo credits
© Copyright Stanley Howe and licensed for reuse under this Creative Commons Licence.
Mental Elf’s strapline includes the claim ‘No misinformation’. If that was only the case for Lisa Burscheidt’s posting about research into the use of music therapy in young people with mental health problems. Her headline ‘Turn on, tune in, drop out! Music therapy no better than usual care in young people’ is such a distortion of the results of the Music in Mind Trial, it would make the sub editor of a red top newspaper blush. Let us quote from the conclusion of the study: ‘This study, through its identification of an improved clinical effect in self-reported communicative and interactive skills of adolescents, and an improvement for both children and adolescents in levels of depression and self-esteem in the short term, provides evidence of the potential of music therapy’. Does that really sound like the researchers were saying that music therapy was no better than usual care for young people?
Some of her criticisms of their methods are also pretty off the wall. She opines that the fact that the participants were not blinded to their study condition was a serious limitation. Of course they weren’t blind to what was going on. It’s one thing slipping someone a sugar pill instead of a cancer drug, but it’s another thing altogether handing a teenager a drum kit and then pretending that you haven’t. Believe us, when a kid gets music therapy, they know they are getting music therapy. The idea that you could somehow fool them that they weren’t simply beggars belief. The important thing was that the researchers gathering the data didn’t know.
According to Lisa, another serious limitation was that they relied on the children and parents involved to gather their results, rather than depending on the expert observations of clinicians. We think that the researchers’ reliance on the people who were actually experiencing mental health problems, or dealing with them as parents, was one of the strengths of the study, for the simple reason that these were the people who knew best, And anyway, how would clinicians go about gathering this sort of information systematically? By using exactly the same sort of psychometric instruments that were used by the Music in Mind researchers!
We could go on and challenge other issues, but we think we have made our point. What is most disappointing for us is that this sort of negativity does little to support the cause of music therapy as an option of choice in the treatment of troubled individuals. To adapt one of Lisa’s captions ‘Another Mental Elf review that fails to find much benefit in randomised controlled trials. Where does this leave patients and policy makers?’
Dear team,
Thanks for taking the time to respond to my blog post. I will try to the best of my ability to reply to the points you raise. Before I start, I would like to get some of the more emotive issues out of the way.
“This sort of negativity does little to support the cause of music therapy as an option of choice in the treatment of troubled individuals.”
I should like to point out that my feelings about the study were not so much about negativity but about disappointment. Personally, I thought that the study involved a good size sample and a set-up that might have led to some really solid evidence but unfortunately it falls short in ways that make it unsuitable as a piece of evidence to use to advocate for music therapy. I should also like to make it very clear that the post is not an expression of my personal emotions, but the result of my using a critical appraisal tool on the trial. It is not my concern, or the concern of this blog, to advocate for one particular therapy. It is my concern to critically appraise research studies in my areas of interest and expertise, CAMHS being one of them.
With that out of the way, let’s start at the beginning. You are not the only ones to criticise the headline of the post. My choice of words for it was more flippant than it should have been and I take full responsibility for that.
You say: “Let us quote from the conclusion of the study: ‘This study, through its identification of an improved clinical effect in self-reported communicative and interactive skills of adolescents, and an improvement for both children and adolescents in levels of depression and self-esteem in the short term, provides evidence of the potential of music therapy’. Does that really sound like the researchers were saying that music therapy was no better than usual care for young people?”
For the purposes of my post, it doesn’t matter what this sounds like. This is *the researchers’* conclusion/interpretation of the study. My post is my critical appraisal of the study, and after reading the study and careful critical appraisal, *I* came to the conclusion that the study did not present convincing evidence for music therapy: The abstract of the paper itself clearly states that there were no significant differences for child or guardian SSIS or family or social functioning at week 13 and that the modest short-term effects on depression scores and self-esteem as well as self-reported improvements in communication and interactive skills that were observed at week 13 were not sustained over time.
With regard to the criticisms of the study regarding blinding and self-report, the paper itself states that taken together, these represent a weakness, and that clinicians should use other methods to gather outcome data systematically:
“Fourthly, outcome measures were all based solely on parent and patient report. Given that parents and patients were not blind to study condition, some other form of outcome (behavioural observations by a clinician) would have helped strengthen the study, and we strongly recommend this be considered for future research.”
My point of view was that since the trial had no blinding, no attention control and relied entirely on self-report, it was highly flawed. Each of these by itself can be considered a methodological flaw, but taken together they mean that the results don’t hold up to critical scrutiny.
The lack of attention control and the use of “usual care” means that the young people in the control group received variable levels of intervention. Some had talking therapies/counselling, some were medicated, some received a combination. It is therefore questionable whether we can consider that the groups were treated equally apart from the intervention. It is becoming more and more common to use some form of attention control in trials for behavioural interventions or therapies and I would have liked to see it used here. I do not understand the rationale for not including one. How many of the participants who were on medication only were waiting for counselling/therapies?
In the worst case, on the one hand we might have a portion of the control group who were on medication, waiting for talking therapies, and getting worse over the trial period, and an intervention group for whom the intervention may have worked because they expected it to work – in other words, whose self-report measures show improvement because they expected improvement.
However, all this is almost negligible in the face of the fact that 38% of the intervention group dropped out, and that only 76 of the 123 children randomised to music therapy (only just over 60%) completed outcome measures and were included in analyses. This simply does not represent convincing evidence for the intervention.
One of the criteria used for appraisals of trials is that a trial should be considered “broken” if there is < 80% follow-up. Where there is large loss to follow-up, lost patients should be assigned to “worst case” outcomes and the results recalculated. The loss is only acceptable if those results still support the original conclusion of the study. As the study itself states, “the high rate of missing data at week 26 introduces a weakness and we caution the interpretation of the analysis at this later time point”. From the supplementary tables I get the impression that the analysis only included those participants for whom data was available at this later point, and that missing data were not accounted for. I would be happy to be corrected on this.
Decision makers ask hard questions, such as: Is the observed small improvement really due to music therapy? Could it also be simply because the young people were provided a space that was neither school nor home, where they were (supposedly) under no pressure and could express their emotions freely? Would another arts therapy work just as well, or better? Does the person who sits in the room with them have to be a music therapist or would we see similar effects if this was delivered by a teacher or a community musician at a fraction of the price? Does the (modest, short-term) improvement justify the cost of this intervention?
Unfortunately the results of this trial cannot answer any of them. What makes this so frustrating is that the data shows that there was a comparatively large very adherent group, who likely found the therapy beneficial in one way or another, but 30% of outcome data for them are missing, which makes it impossible to recommend the intervention with confidence.
I hope that this clarifies my post. As mentioned above, I am happy to be corrected.
Last but not least, you're welcome to address me directly rather than referring to me by my first name only. Referring to women in this way is fraught with problems.
Mmm mmm – complicated! Reading the comments, especially so.
Music and arts and the interaction with our human journey and our emotions – and brains, to be that reductive but nonetheless fascinated with the neurological science – is an enormous topic.
Too big for this sort of study model, really.
Emotions are an important way we condense complex perceptions, music and art connects into that kind of circuitry if I gather right.
And I think it is through this unverbalised channel, in attempts at circumspection, that sharp emotions and fuzzier evocations and qualifications and political pleas have come into the narrow constraints of making and assessing this kind of study.
Some words that weren’t nice or particularly scientific or justified. Opined, in reference to discussion of controls. That reply goes over my head a bit, but I know controls are important and especially in currently favoured models of clinical research. You need to try to compare like for like in a large population, with the particular variable of focus the most likely variable to account for any significant difference. And that’s to understate the process since I understand it at a very basic level.
Though I sometimes wonder if it’s a bit overblown and up itself as a set of requirements I also appreciate that we need to know that we know what we think we know. That we can trust inferences about cause and effect. You all care about that. And about music.
I’m contemplating how to play peacemaker or at least make a measured contribution. I think music therapy has a lot of value or potential value in special education but this area was excluded from the study, the focus was on mental health. I’m supportive of arts therapy in general, have emotions about it, but I think it can be done well and badly not just done and I’m all for studying and refining therapeutic aspects though finding the magic is something of an art, and I think the general chance for arts exposure is not something I hate public money going to if it can’t be proven beneficial in mental health terms. I think there should be practice rooms everywhere. Unless we find out that actually, our long held almost globally held human intuition is wrong and music is poison to the soul.
It would be interesting to see more analysable data about how the participants used language or emotional expression and how this changed over time. Surely this is the mechanism and is important. And also interesting and less threatening to monitor.
On the other hand, that may be further caging the nightingale. Or a Victorian zoological sample collection, which has value and severe hateful costs too. Turning mental health and artistic experience under the microscope in such formalised ways risks turning well-being into an exam and clinical research in therapy into the science of cramming.
As an amateur poet/musician/artist, with SpLDs and children with autism and/or autistic like SLDs/brain damage with MH difficulties and challenging behaviour, a former service user myself who still has regular suicidal ideation irrespective of ups and downs, I can say life is a mixed bag and should be. But that doesn’t mean we shouldn’t try to examine and shift our parameters. And if reading all of a Daisy Goodwin anthology or writing while depressed did or didn’t help, and it can certainly intensify things, then that doesn’t mean that I shouldn’t have done those things.
This goes to the heart of what well-being is and who mental health intervention is for. Beyond that, there’s a lot of complex questions. There’s the theory that happiness is a journey, it’s resolution of problems, not a state of mind. And how can we tell what solutions, skills, resolutions, new vistas and things worthwhile for their own sake we’re investing in with arts and how long the follow up period should be. Other studies, I seem to remember, show some general benefits or neurological changes without significant difference in outcome. I don’t know how you could link this study with such
others or if you did.
The drop out rate and the callously triggering nature of self report, which I can vouch for as a service user, should be the discussion where we find fertile, useful common ground to further develop some philosophical and practice questions raised by the study, for science, therapeutic questions and the way that science in general and clinical research and practice in particular interface with society. I think there are lots of of bigger issues here we need to be more conscious about without getting lost in the details.
Obviously, mindfulness is the panacea for everything, clinicians and allied professions as well! Did you not get the memo!? Muhahahaha!
As far as I am concerned until professional music therapy training respects scientific method and evidence-based medicine, this debate will never get resolved.