Previous meta-analyses in the field of depressive disorders have yielded mixed conclusions about which the most efficacious treatment is. Within the process of any therapy, there are universal features such as; the therapeutic relationship, rationale for treatment, a patient and therapist belief in treatment, a confidential and respectful space for the patient and so on. However, individual to each psychotherapy there will also be specific factors such as for example; the use of behavioural experiments in CBT or the focus on the patient/therapist relationship in psychodynamic therapy.
Braun, Gregor & Traun (2012) set out to complete four tasks:
- Update evidence on bona fide psychotherapies for depression
- Clarify and investigate the status of non-CBT bona fide therapies in comparison to CBT
- Compare two meta-analytical approaches (conventional random-effects model and a network meta-analysis)
- To identify moderators that may explain relative efficacy of different psychotherapies
Bona fide therapies were identified by three criteria:
- That therapists were trained to provide the treatment and held adequate qualifications
- That therapists met face to face with the person to deliver an individualised treatment (i.e. not following a standard protocol)
- That the treatment contained “psychologically valid components”
Methods
Using a systematic search strategy the reviewers identified 53 papers which were appropriate for inclusion. Studies included were published between 1977 and 2012, including 3,965 patients of which 70.6% were female.
Criteria for inclusion were that the studies had to compare efficacy of psychotherapy treatment for depressive disorder with another psychotherapy treatment in a randomised trial, and that at least two treatments that were compared were bona fide.
Study population (comorbidity, age, gender, clinical diagnosis etc.), treatment (number and length of sessions, therapy format, therapist training etc.) and study characteristics (randomisation, study quality, researcher allegiance etc.) were considered as possible moderators and coded by reviewers (where double coded there was a 93% agreement). Study quality (assessed using a risk bias assessment tool) and researcher allegiance were only coded once.
Independent effect sizes (converted to Hedges g in order to adjust bias due to small sample size; 0.20 is considered small, 0.50 medium, and 0.80 large) were calculated for the five outcome measures:
- Patient rated Beck Depression Inventory (BDI)
- Clinician rated Hamilton Depression Rating Scale (HDRS)
- Combined outcome measure based on mean of patient rated BDI and clinician rated HDRS
- Ratio of odds (OR) of remission
- OR of dropout
Results
Meta-analysis I included only therapies where there were 5 or more comparisons available and found:
- There is no strong indication of superiority of CBT, Behavioural Activation (BA), Psychodynamic Therapy (DYN), and Interpersonal Therapy (IPT) (range of g= -0.09 to 0.19) however Supportive Therapies (SUP) according to Rogerian theory were shown to be less efficacious than other therapies (g=0.26)
- There was no evidence of therapy (CBT, BA, DYN or IPT) superiority in relation to remission rates (OR range: 0.87 to 1.02; OR>1 indicates higher likelihood of remission), however Supportive Therapies (SUP) were less efficacious (OR=0.65, p<0.01)
- Moderator analysis:
- Participants over the age of 60 appeared to benefit more from BA than other treatments and BA was more efficacious when delivered in an individual rather than group/couples setting
- According to clinician ratings, CBT was more efficacious than other therapies for females
- CBT was more efficacious than other treatments when therapy lasted 90 minutes or longer, whereas BA was more efficacious when therapy sessions lasted less than 90 minutes. Longer sessions were more likely to be associated with a group format delivery
Meta-analysis II allowed for analysis of previous grouped together SUP therapies and using CBT as a comparison treatment:
- None of the comparisons between CBT and other therapies reached nominal significance, however reviewers commented on trends which indicated some efficacy in “other therapies” including cognitive behavioural analysis system of psychotherapy (CBASP) and problem solving therapy (PST), albeit based on a small number of comparisons
- Comparison of results from meta-analysis I and II indicate different results across patient, clinician and combined outcome measures on CBT vs. SUP, BA, DYN and IPT. Results show larger effect sizes in the network meta-analysis (II).
Conclusions
The authors concluded:
Our results suggest that the dodo bird verdict seems to be the right answer for the wrong question. Even though it seemed mostly corroborated at the aggregate level, there appear to exist a number of differential effects in efficacy between bona fide treatments of depression at a finer level.
The results of the meta-analyses showed small effect sizes for the differences between psychotherapies, effectively suggesting that (of the therapies included in this review) there is not one therapy which stands out as yielding better outcomes. However, results do throw up some interesting questions about the moderating effects of; participant age, therapy format, patient gender, nature of disorder and comorbidity.
The limitations of this review impede any firm conclusions being drawn:
- There were not enough studies to conduct separate meta-analyses for each type of treatment. Overall there were three times as many CBT comparisons (k, number of comparisons=41) than any other therapy BA (k=16), DYN (k=11), IPT (k=10), SUP (k=17)
- As well as a small number of study comparisons there was variability within the sample which is relatively unclear (e.g. patient diagnosis and comorbid presentations) and some potentially relevant details are not reported (e.g. depressive episode or duration of illness)
- The use of network meta-analyses may not have been appropriate given the diversity of the study samples as it is more likely to make assumptions about the data when looking at both direct and indirect comparisons
In essence the most comprehensive conclusion to be drawn is that more research into psychotherapies for depression is needed. This would allow for more therapy specific meta-analyses which might yield results that have more clinical significance. This is no mean feat however, after all, this review highlights the challenge of being able to define and measure the many nuances of therapy and their relative contribution to each therapy session.
Link
Braun SR, Gregor B, Tran US (2013) Comparing Bona Fide Psychotherapies of Depression in Adults with Two Meta-Analytical Approaches. PLoS ONE 8(6): e68135. doi:10.1371/journal.pone.0068135
@Mental_Elf Classic conclusion – dodo bird and more research needed = a waste of space. No mention of Human Givens http://t.co/HcORzvp8iw
“Everyone’s a winner, all must have prizes!” but which psychotherapy for depression wins, if any?: Previous me… http://t.co/uZ6YM8hg4p
http://t.co/D28heixypa #mental #health #mentalhealth #psychotherapy #therapy #counseling #therapy #tokyo #japan #worldwide #depression
@Mental_Elf Sorry to keep banging on, but, would have to say CBT because there are no other choices!
@getmindfuel’s debut blog about a @PLOSONE review comparing bona fide #psychotherapies of #depression in adults http://t.co/H6kQncrbLe
New meta-analysis searches for clarity about which #psychotherapy for #depression is best http://t.co/H6kQncrbLe
@Mental_Elf Disappointing. Why slavishly summarize art, & not produce any perspective or critique? & the conclusions on BA for old age TBC
@Mental_Elf simply don’t follow.
Groundhog Day for the Dodo Effect. Another big meta analysis suggests no type of therapy beats others for depression http://t.co/N8yIz5sYib
@PsychiatrySHO Couldn’t agree more. And why not say so in the @Mental_Elf blog?
Review finds that behavioural activation is more effective than other talking treatments for depressed older people http://t.co/H6kQncrbLe
@Mental_Elf In normal language are we saying getting out and about ,joining in etc is good?? why do we have 2 make it sound revolutionary!!
@Mental_Elf That’s not what article shows. Effect confounded w group / individual thpy. And the 2 refs (38 & 39) show nothing – no RCTs.
@Mental_Elf interesting depression study but why assume one form of therapy would be best for all individuals with depression?
No therapy a clear winner in treatment of #depression. http://t.co/jl7CPO8P8c
Don’t miss: “Everyone’s a winner, all must have prizes!” but which psychotherapy for depression wins, if any? http://t.co/H6kQncrbLe
@Mental_Elf i am missing acceptance and commitment therapy.
@Mental_Elf Believe we are looking in the wrong place. Approach is of little importance compared with other factors
“Everyone’s a winner, all must have prizes!” but which psychotherapy wins, if any? Via @Mental_Elf http://t.co/mZ5K7WqkeR
RT @APEAL_Lab: Psychotherapy for #depression: anything is better than nothing. Every little helps. Nice summary from @Mental_Elf. http://t.…
Mental Elf: “Everyone’s a winner, all must have prizes!” but which psychotherapy for depression wins, if any? http://t.co/Q41ZYrJAiy
Good summary of ‘Dodo bird’ meta-analysis of treatment for depression, but what effect will it have on the debate? http://t.co/rbX7JLsvhS
I’ve appreciated the comments and discussion generated from this blog they’ve been really interesting and thought provoking. My statement at the end of the blog that “more research into psychotherapy research is needed” was not intended to be flippant, and on reflection i should have elaborated!! To clarify I do think that the findings, despite being inconclusive, highlight some potentially interesting avenues to explore in the future such as; how better understanding of moderating effects of individual therapies might help us to improve the availability of a variety of therapies. Whether we agree or not the future of treatment availability (in the NHS) is about streamlining treatment pathways and that places greater emphasis on the evidence base and the detail of the findings which are made available, which in turn could minimise the risk of a one-size fits all approach in relation to treatment availability (as some might argue has happened with CBT). Hence @suepullan- “why one therapy for all?” is very relevant! Maybe the dodo effect keeps being replicated for the very reason that one therapy might work better than another for some people, but that’s not to say that the next person would also find the same benefit, and neither outcome necessarily negates the effectiveness of that treatment. In our perpetual pursuit and frustration of trying to find “the winner” we might dismiss some of the value of what gets uncovered along the way; just as in therapy if we focus so much on the outcome or the goal then the opportunity to learn from the process (which is inter-relational and not just being pleasant) can be lost. In essence, whilst its a reference which is continually made in this field of research (and the authors started with this in their paper, hence my reference) I’m not sure I find its repetitiveness intolerable because if interesting research questions are being asked then what is found out adds to the richness of what we know.
Most of the studies included both a therapist rated and patient rated scales, namely the HRSD and BDI. I agree with comments which suggest that both types of measurements have limitations, particularly when outcomes are measured in this way however it is also one of the major challenges of trying to operationalize outcomes in a bid to strengthen the evidence-base of these therapies. These two outcome measures in many respects are comparable as to what they are measuring however how appropriate are these tools across different models of therapy? They are particularly suited to CBT, which is also the most represented in this study but would outcomes overall have been different if wellbeing measures had been used or if level of activity were considered or even if measures of therapeutic rapport had been taken; in essence shouldn’t we be taking a more holistic view when it comes to outcomes? I personally think we should. Patients, in my opinion are and should be our primary indicators of outcomes because it is their subjective distress which often leads them to be in therapy in the first place. Having said this, inevitably bias will be present across all types of measurement and where blinding is possible I agree that it is important but even so the issue would exist anyway.
I’d be happy to invite wider discussion about this- clearly people will have different views and all are welcome- and as has been pointed out there are therapies and models which even considered in the study!
@henrystrick @PsychiatrySHO Care to respond to @GetMindfuel’s new comments on her blog? http://t.co/Q2AbqbSOVL
“Everyone’s a winner, all must have prizes!” Dodo verdict: which psychotherapy for depression wins, if any? http://t.co/2bbqghk2hi
Thanks to all (especially Henry Strick and Alex Langford) for the interesting discussion generated on twitter recently. I think some interesting points were made which i would like to reply to on the blog discussion and hopefully invite some wider contributions from others as well.
Before i do however i just wanted to clarify i understood the points raised:
1) It was felt there was a general lack of critique on my behalf, particularly about the replication of the “dodo bird effect”.
2) The issue of outcome measurement in psychotherapy seems to have been raised as a general discussion point (which my blog did not address)
3) Henry, this relates specifically to one of your comments “That’s not what article shows. Effect confounded w group / individual thpy. And the 2 refs (38 & 39) show nothing – no RCTs.” I’m afraid i don’t quite understand what you are referring to, so if you could clarify then hopefully i will be able to respond.
4) The conclusion “more research into psychotherapy for depression” wasn’t felt to be adequate.
Best wishes
Sammy
Hi Sammy,
On the points you raised:
1. Yes indeed. I felt the reference to the dodo bird in the title wasn’t good enough, just rehashes its existence. I didn’t think that the final paragraph’s discussion in the article was at all sufficient – what they said about the dodo bird effect not having been fully confirmed was not substantiated by anything they wrote. I think this is a common problem, that by now everyone has heard about that somehow (partly because of the catchy name), but people don’t know “what to do about it”. That’s where I think Mental Elf good give a lead, by having a general vision about what to do about the countless research articles that repeat the dodo conclusion. Lambert, M. (2013). Bergin and Garfield’s Handbook of Psychotherapy and Behavior Change (6th ed.) – I do hope you have access to this / are familiar with it – is one of the best and most recent trying to move the field of psychotherapy research “beyond the dodo”…. Basically, all psychotherapy research from now on should take the dodo result as a starting point, a background against which to report – in the same way that nobody in physics does research or reports the reconfirmation of Newton’s laws of mechanics. This article didn’t do that; so you as Mental Elf could have suggested that they should have taken the dodo result more as a given than as a study outcome. Or possibly they shouldn’t even have published…. Replication of original research is very useful; replication of meta-analysis is I think rather sterile.
2. It depends a bit whether you think that the discussion about outcome measurement is inconclusive or has led to some firm conclusions. This is related to the important question whether therapist effects in general are under-researched. If you read Lambert (2013) Ch.8, and Chs 8 and 14 of Duncan et al (2010), The heart and soul of change, you and the whole Mental Elf could reach the conclusion that therapist factors need a lot more emphasis than they get, and that this implies that outcomes that are mainly based on therapist evaluation of the outcome are suspect and not good enough (as they can’t be unbiased about themselves). This too, in turn, for this particular study, could have led you to state that is was unsatisfactory that despite their separate reporting of the self-rated and the therapist-rated outcomes, they did not do more about this difference.
3. In the Discussion, paragraph 6, the article states that BA is “more efficacious than other treatments” for older patients, and refers to references 38 and 39. These references don’t say that at al, they are just non-controlled studies showing that certain older patients improve when given BA (and also, the definition of BA in general I think compares very doubtfully with the “treatment” described in these two studies).
More importantly, this same paragraph starts with the statement that “behavior activation therapy was more efficacious than other treatments in samples of older patients”. But this is based on the paragraph in Moderator analysis under “participant age and treatment format”. However, the numbers there don’t support that strongly formulated conclusion. Firstly, it states plainly that the better performance of BA is not there for the self-report measure. Secondly, it states a bit furrther on in “participant age and treatment format” that “format” (by which they mean the effect of group therapy vs individual therapy) and age effects were confounded. They go on to explain that when they estimated the effect of age and format simultaneously, the positive effect of BA for old age disappeared. The sixth paragraph in discussion however completely ignores this. (Is it possible that the discussion was written at a stage that the refinement of the confounding had not been established yet?) Anyway, this is a clear hole in their argument that you could have picked up.
In general, I can see that this would have required rather precise reading of the article. However, it was worth checking out the complete details of their proof of this conclusion, as this was the only concrete example that they could show that seemed to support their conclusion of “doubts about the dodo verdict”. If you agree that in fact the point about BA for older patients is not made, they did not manage to give a single exception to the dodo verdict. And this leads back to the general question: Do we need more meta-analytic studies to see if the dodo verdict can be undermined?
4. Psychotherapy for depression is about the most-researched and the most-repeated research area of any in psychotherapy. Especially exactly the kind of studies that were (again!) meta-analysed in this article. The challenge is to explain why people keep repeating the same kind of study, with the same kinds of little problems that always keep returning. In the article, and in your blog, rather than saying: let’s do more research about psychotherapy including CBT and other types for depression, there could have been a call for: “Meta-analysis always produces the same results of the endlessly repeated studies. Why don’t researchers research something different, in order to take the field forward? Why don’t they go further in understanding therapist effects, or study which factors are most effective in “therapy”, or do more “taking-apart” work in existing therapy models?”
I hope this gives you a bit more “meat” as to what I was after, and where I thought you could have done more. It’s not to say that I thought your blog was a bad blog – it was careful and diligent, and better readable than the original article. But I’d hoped for “more” – of which perhaps something is coming through in the discussion.
Good to “meet you” virtually; hope to see more of your blogs.
Best wishes,
Henry
Hi Henry,
Thank you for your reply- I need to go back and have a look at the paper which I haven’t yet had a chance to do and i can’t say that i’m familiar with that book although its helpful to know what you were referring to with respect to the dodo point as the starting point. I also see what you mean about replication about a meta-analysis vs the possibility of carrying out research into different aspects of therapy- I too would be very interested in more studies looking at specific aspects of therapy. A colleague had explored the possibility of such research a couple of years ago and one of the challenges she had found was that therapists were quite reluctant to take part when the therapy involved being observed in sessions, I suspect as it felt exposing however I do wonder if replication is often the result of these types of limitations rather than a lack of interesting research questions.
Its certainly been helpful to receive your insights and comments, and appreciate the time you’ve taken to reply as well.
Best wishes
Sammy
[…] against each other. To find out more about the problem of comparing psychotherapies, try reading Sammy Man’s blog from January […]