Objective To show that overpowered trials claim statistical significance detouring clinical relevance and warrant the need of superiority margin to avoid such misinterpretation.
Design Selective review of articles published in the New England Journal of Medicine between 1 January 2015 and 31 December 2018 and meta-analysis following Preferred Reporting Items for Systematic Reviews and Meta-Analyses checklist.
Eligibility criteria for selecting studies and methods Published superiority trials evaluating cardiovascular diseases and diabetes mellitus with positive efficacy outcome were eligible. Fixed effects meta-analysis was performed using RevMan V.5.3 to calculate overall effect estimate, pooled HR and it was compared with mean clinically significant difference.
Results Thirteen eligible trials with 164 721 participants provided the quantitative data for this review. Largely, the primary efficacy endpoint in these trials was the composite of cardiovascular death, non-fatal myocardial infarction, unstable angina requiring rehospitalisation, coronary revascularisation and fatal or non-fatal stroke. The pooled HR was 0.86 (95% CI 0.84 to 0.89, I2=45%) which was lower than the mean clinically significant difference of 0.196 (19.6%, range: 0.09375–0.35) of these studies. There was a wide 95% CI in these studies from 0.56 to 0.99. The upper margin of CI in most of the studies was close to the line of no difference. Absolute risk reduction was small (1.19% to 2.3%) translating to a high median number needed to treat of 63 (range: 43 to 84) over a follow-up duration of 2.95 years.
Conclusions The results of this meta-analysis indicate that overpowered trials give statistically significant results undermining clinical relevance. To avoid such misuse of current statistical tools, there is a need to derive superiority margin. We hope to generate debate on considering clinically significant difference, used to calculate sample size, as superiority margin.
- clinical pharmacology
- clinical trials
- overpowered trials
Statistics from Altmetric.com
Contributors NG and SM hand-searched the articles and independently reviewed the articles. Disagreements were discussed with NS. All three authors contributed in data analysis and manuscript preparation.
Funding The authors have not declared a specific grant for this research from any funding agency in the public, commercial or not-for-profit sectors.
Competing interests None declared.
Patient consent for publication Not required.
Provenance and peer review Not commissioned; externally peer reviewed.
Data availability statement There are no data in this work.
If you wish to reuse any or all of this article please use the link below which will take you to the Copyright Clearance Center’s RightsLink service. You will be able to get a quick price and instant permission to reuse the content in many different ways.