The NHS has entered the era of GP-led commissioning of services, and providers will be required to submit data on key indicators such as quality of care and survival rates.

This echoes some aspects of a 2001 report by the Institute of Medicine in the USA Crossing the Quality Chasm, which emphasised the need to “align financial incentives with the implementation of care processes [and] patient outcomes”.

Since this report was published, pay for performance (P4P) has been a topic of considerable interest. P4P refers to the provision of financial incentives to achieve pre-specified criteria, such as measures of care delivery or patient outcomes. However, clearly research is required to establish whether P4P delivers genuine and sustained improvements in these outcomes.

A recent study investigated P4P in the context of treatment for adolescent substance use disorders. The researchers were interested in whether P4P improved treatment implementation and effectiveness of an evidence-based behavioural treatment called the Adolescent Community Reinforcement Approach (A-CRA).

Methods

A total of 34 community-based treatment organisations received funding to implement A-CRA, of which 29 were randomised in a cluster randomised trial. This design means that therapists within each organisation are either all receiving the P4P incentive, or not, to prevent compensatory rivalry or resentful demoralization. In addition to the implementation-as-usual procedures at all participating centres, therapists at organizations randomized to the P4P condition had the opportunity to earn monetary bonuses for the achievement of two pre-defined performance measures of treatment implementation. In total, 15 organisations (60 therapists and 634 patients) were allocated to the implementation-as-usual condition, and 14 organisations (60 therapists and 539 patients) to the P4P condition.

The primary outcomes were measures of treatment implementation and treatment effectiveness. Treatment implementation was assessed independently using audio recordings of treatment sessions. These were used to determine how many pre-specified A-CRA procedures and how many treatment sessions were delivered. Treatment effectiveness was assessed as patient-level remission status (i.e., whether they reported no substance use, abuse or problems of dependence in the past month while living in the community).

Results

• The results indicated that therapists assigned to the P4P condition were more likely to demonstrate A-CRA competence compared to those in the implementation-as-usual condition (24.0% vs 8.9%, P = 0.02).
• Similarly, patients in the P4P condition were more likely to receive the target A-CRA (defined as at least 10 out of 12 procedures and 7 sessions) compared to those in the implementation-as-usual condition (17.3% vs 2.5%, P = 0.01).
• However, in terms of patient-level remission status, there was no difference between patients in the P4P condition and those in the implementation-as-usual condition (41.8% vs 50.8%, P = 0.25).

Conclusions

The authors conclude that:

P4P can be an effective method of improving implementation of evidence-based treatments in applied settings.

However, they also note that there was still room for improvement in the overall levels of A-CRA competence and target A-CRA, even within the P4P condition.

The authors argue that this may have been due to poor compliance with the procedure required to validate the outcomes (e.g. recording of sessions). However, it’s also possible that since therapists knew they were being monitored their performance may have been higher than usual – this is a known problem in health service research of this kind.

Clearly P4P seems to improve treatment delivery, but the absence of any improvement in patient outcomes suggest that metric-based assessments of delivery quality may not capture all of the factors that contribute to patient outcomes. The authors argue that this may have been because patient outcomes were better than usual across both conditions, making it harder to show superiority in the P4P condition. This may be true, but clearly it is these outcomes which ultimately matter, and if P4P does not lead to an improvement in these then we should be cautious.

Link

Garner, B.R., Godley, S.H., Dennis, M.L., Hunter, B.D., Bair, C.M.L. & Godley, M.D. (2012). Using pay for performance to improve treatment implementation for adolescent substance use disorders. Archives of Pediatric and Adolescent Medicine, 166, 938-944. doi:10.1001/archpediatrics.2012.802 [Pubmed abstract].