My grandfather had a nearly religious conviction of the value of physical activity. No weekend was complete without a strenuous bout of outdoor exercise, regardless of the weather.  New England gets cold and damp, and his five children would prepare themselves for winter excursions with some reluctance. ‘Healthy body, healthy mind!’ my grandfather would say, exhorting them to bear up and show some of the fortitude of their Puritan ancestors.  It was perhaps fortunate that their thick scarves muffled any replies.

‘Healthy body, healthy mind’ is from the Latin mens sana in corpore sano, ‘a healthy mind in a healthy body’. It was with some surprise that I learned that the original source doesn’t propose a causal link; the writer is merely listing the things that one should value in life. My grandfather’s interpretation, however, is a widely held belief; in fact, it is tacitly supported by none other than the World Health Organization. In the late 1980s, the WHO introduced the concept of the ‘Health Promoting School’, an institution that supports the development of healthy lifestyles as well as education. The Health Promoting School framework has since been incorporated into many countries’ educational policies.

Earlier this year, the Cochrane Collaboration released a systematic review of ‘the effectiveness of the Health Promoting Schools (HPS) framework in improving the health and well-being of students and their academic achievement’ (Langford et al, 2014).

Methods

An international panel of researchers conducted an extensive and systematic search to identify cluster-randomised controlled trials of HPS interventions aimed at students aged 4 to 18.  ‘Cluster randomisation’ means that randomization takes place at the group level; in this case, schools or districts in which interventions were implemented were evaluated against comparable schools or districts in which no intervention took place.  In order to qualify as ‘HPS’, an intervention had to include the following:

• Introduction of health education into the curriculum
• Changes to the school’s ethos or environment or both
• Engagement with families or communities, or both

Sixty-seven trials, including a total of 1,443 schools and districts, were found eligible for the review.  The researchers grouped the trials according to health topic and approach used. Methodological quality was assessed using GRADE (PDF), which is a widely used standardized system for determining the overall quality of a group of studies. Risk of bias was also considered.

Results

Most studies were set in North America (29), Europe (19), or Australasia (11). Interventions were slightly more likely to target younger children (<12). Length varied from eight weeks to two years; the authors noted that shorter interventions were more likely to be aimed at physical activity or nutrition, whereas interventions focusing on substance use, violence, and sexual or mental health tended to be longer.

The overall quality of the evidence was low to moderate, suggesting that additional studies could lead to large changes in the researchers’ conclusions.  High attrition and a reliance on self-reported data created a risk of bias in many of the trials.

That said, the authors found that Health Promoting Schools had a:

Positive effect	Little effect	No effect	Insufficient evidence for conclusions
For the following intervention targets:
Body mass index (BMI) Physical activity Physical fitness Fruit and vegetable intake Tobacco use Being bullied	Standardized body mass index (zBMI)	Fat intake Alcohol use Drug use Mental health Violence Bullying others	Academic, attendance, or school-related outcomes

 

Conclusions

The authors conclude:

Overall, we found some evidence to suggest the HPS approach can produce improvements in certain areas of health, but there are not enough data to draw conclusions about its effectiveness for others. We need more studies to find out if this approach can improve other aspects of health and how students perform at school.

Summing Up

The authors’ conclusions, conservative though they are, are already being used to argue for greater investments in Health Promoting Schools. In light of the gaps identified in the review, it is important that some of this funding is allocated to rigorous evaluation. In my view, three areas are particularly in need of further research:

1. Academic outcomes. According to the authors, ‘the HPS framework is based upon a recognition of the intrinsic link between health and education’, yet few studies examine this link. Implementing a health promotion curriculum in schools implies an unavoidable diversion of resources from traditional educational activities. Evidence suggests that health and academic performance are complementary goals rather than competing ones, but more research is needed.  The presence of a correlation between better health and improved academic outcomes would strengthen the business case for Health Promoting Schools, particularly if randomised controlled trials suggested causation. If no correlation were detected, that too would be worth knowing.
2. Adverse effects. Do obesity prevention campaigns increase disordered eating?  Could anti-tobacco initiatives lead more students to cigarettes?  Small adverse effects can be significant at the population level, so it’s important to monitor them when large-scale implementations are being considered.
3. Long term outcomes. Very few trials in this review (10) included follow-up measures. As the authors write, ‘While interventions may be able to produce short-term changes in behaviours or health outcomes, unless these prove sustainable they are likely to be of little public health importance’ (p. 32).

The suitability of randomised controlled trial designs for evaluating health promotion initiatives and other complex interventions has been questioned (Green, 2000; Tones, 2000 (PDF); WHO European Working Group, 1998 (PDF)). Opponents argue that standardising an intervention in an environment with as many uncontrollable variables as a school is neither feasible nor appropriate. Hawe et al. (2004) (PDF) offers, in my opinion, a sound counter-argument: Randomised controlled trials are suitable, provided we are willing to moderate our definition of ‘standardised’. It’s unnecessary to distribute precisely the same posters and teach exactly the same training sessions in all schools, and one would be foolish to expect good results from doing so. Tailoring an intervention to its target population is an accepted factor in its success.  Therefore, “in complex interventions, the function and process of the intervention should be standardised, not the components themselves”(Hawe et al., 2004, p. 1563).  This approach avoids placing senseless restrictions on program developers while still allowing us to evaluate interventions in a consistent manner and make some broad statements about which outcomes appear most susceptible to change.

A lengthier discussion of the use of randomised controlled trials with complex interventions appeared in a recent report by Haynes et al. (2012) (PDF).  The authors make a persuasive case for the inclusion of randomised controlled trials in policy development. Among their points:

• RCTs reduce our dependence on professional opinion and expert intuition, which have been shown to be more fallible than one might expect
• RCTs help us determine whether individual components of an intervention are essential or redundant, leading to long term cost savings
• RCTs can reveal adverse effects that might otherwise have gone unnoticed, preventing us from unintentionally harming hundreds or thousands of people

Other types of evaluation are, of course, also necessary; the perceptions of students, educators, and administrators are essential to gaining insight into how an intervention worked, or why it didn’t work, which is infinitely useful for program development.  It doesn’t undermine the value of qualitative evaluation methods, however, to state that the randomised controlled trial (even the bastardised form used in complex interventions) is deserving of a place.

Links

Langford R, Bonell CP, Jones HE, Pouliou T, Murphy SM, Waters E, Komro KA, Gibbs LF, Magnus D, Campbell R. The WHO Health Promoting School framework for improving the health and well-being of students and their academic achievement (Review). Cochrane Database of Systematic Reviews 2014, Issue 4. Art. No.: CD008958. DOI: 10.1002/14651858.CD008958.pub2.

Reviewing Evidence Using Grade (PDF). Institute for Clinical Systems Improvement, n.d.

Green J. The role of theory in evidence-based health promotion practice. Health Educ Res. 2000, 15(2): 125-129.

Tones K. Evaluating health promotion: a tale of three errors (PDF). Patient Education and Counseling 2000;39(2-3):227–36.

WHO European Working Group on Health Promotion Evaluation. Health promotion evaluation: Recommendations to policy-makers (PDF). Copenhagen: World Health Organization, 1998.

Hawe P, Shiell A, Riley T. Complex interventions: how “out of control” can a randomised controlled trial be? (PDF). British Medical Journal 2004; 328:1561-1563.

Haynes L, Service O, Goldacre B, Torgerson D. Test, learn, adapt: Developing public policy with randomised controlled trials (PDF). London: Cabinet Office Behavioural Insights Team, June 2012.

Bonell, C. Why schools should promote students’ health and wellbeing. British Medical Journal 2014; 348:g3078.