The coronavirus pandemic has highlighted many of the strengths and weaknesses in trying to develop evidence-based healthcare strategies from the huge amount of data created in the past 8 months. The most obvious strength is the rapid sharing of research papers via the preprint servers; the limitations are a lack of high-quality peer review, or research methodology in the rush to publish. In a recent article Aronson and co-workers looked at the large number of Covid-19 drug trials and the lack of  suitable blinding/masking which is essential in reducing the effects of bias within a trial. The authors found that out of 142 trails only 36% were masked (Aronson et al., 2020) raising important ethical issues about bias and unreliable reporting of study results.

The importance of blinding/masking has been highlighted in an oral health review by Saltaji and co-workers (Saltaji et al., 2018) that showed studies that did not mask both participants and investigators gave treatment effect size estimates that were, on average higher than masked studies. This review is described below. Their research questions were:

• Do oral health randomised controlled trials (RCTs) with adequate blinding of participants, outcome assessors, and health care providers yield different treatment effect sizes (ESs) than trials with lack or unclear blinding?
• Do specific non-methodological meta-analysis characteristics (e.g., dental specialty, type of treatment, type of outcome [objective vs. subjective], magnitude of the treatment ES estimate, heterogeneity of meta-analysis) modify the association between blinding and treatment ES estimates?

Methods

This paper was part of a larger oral health study regarding the methodology used in randomised control trials. Their protocol was registered on PROSPERO (CRD42014014070).

The authors searched six electronic databases (PubMed, Medline, Embase, ISI Web of Science, Evidence-Based Medicine Reviews–Cochrane Database of Systematic Reviews, and HealthSTAR) from database inception to May 2014. The search was not limited to the English language.

Two independent assessors extracted the relevant data. Study quality was assessed on the different levels of study blinding and whether they were present/absent or unclear. Meta-analysis was undertaken using the raw data extracted from the individual studies with the summary estimate being the difference in ESs plus 95% confidence intervals. Subgroup analysis was undertaken in a similar fashion.

Results

• 64 systematic reviews (32 Cochrane and 32 non-Cochrane reviews) satisfied the eligibility criteria for the present report. A total of 540 trials analysing 137,957 patients were considered for this study:
  • Periodontics (36 reviews; 271 trials)
  • Dental public health and paediatric dentistry (10 reviews; 145 trials)
  • Oral medicine and pathology (11 reviews; 80 trials)
  • Oral and maxillofacial surgery (4 reviews; 26 trials)
  • Orthodontics and dentofacial orthopaedics (2 reviews; 12 trials)
  • Restorative dentistry (1 review; 6 trials).
• Risk of Bias – Blinding of patients was judged as adequate (low risk of bias) in 71.5% (n = 386) of the trials, and blinding of the outcome assessment was judged as adequate (low risk of bias) in 59.4% of the trials.
• Quality assessment – Only 33.5% (n=181) of studies were described as double blind.
• Trials with inadequate patient blinding had significantly larger treatment ES estimates (difference = 0.12, 95% confidence interval 0.00 to 0.23, p = 0.046).
• Trials with a lack of blinding of both patients and assessors (difference = 0.19; 95% CI: 0.06 to 0.32)
• Trials with a lack of blinding of patients, assessors, and care-providers concurrently (difference = 14; 95% CI: 0.03 to 0.25).
• Subgroup analysis stratified by other characteristics of meta-analyses (heterogeneity of meta-analysis, type of outcome, and dental speciality) was not statistically significant for any of the characteristics.

Conclusions

The authors concluded: –

We found significant differences in treatment ESs between oral health RCTs based on lack of patient and assessor blinding. RCTs that lacked patient and assessor blinding had significantly larger treatment ES estimates. Treatment ES estimates were 0.19 and 0.14 larger in trials with lack of blinding of both patients and assessors and blinding of patients, assessors, and care-providers concurrently. No significant differences were identified in other blinding criteria. Future meta-epidemiological assembling of a greater number of meta-analyses and trials that takes other biases and different degrees of blinding into account is needed.

Comments

This was a well-constructed methodological meta-analysis demonstrating the importance of including blinding wherever it is possible within a study design to reduce the effects of bias. Poorly designed trials can overestimate benefits and therefore obscure harms to patients. Good methodology must not be sacrificed on the alter of producing results with amplified effect sizes in an attempt to get published as this not only represent bad science it is also unethical (WHO, 2014).

References

ARONSON, J., DEVITO, N. & FERNER, R. 2020. The ethics of COVID-19 treatment studies: too many are open, too few are double-masked [Online]. Oxford COVID-19 Evidence Service. Available:  [Accessed 6th July 2020].

SALTAJI, H., ARMIJO-OLIVO, S., CUMMINGS, G. G., AMIN, M., DA COSTA, B. R. & FLORES-MIR, C. 2018. Influence of blinding on treatment effect size estimate in randomized controlled trials of oral health interventions. BMC Med Res Methodol, 18, 42.

WHO. 2014. Ethical considerations for use of unregistered interventions for Ebola virus disease: Report of an advisory panel to WHO [Online]. Available:  [Accessed 6th July 2020].