Abstract
Background
Many articles in the surgical literature were faulted for committing type 2 error,
or concluding no difference when the study was “underpowered”. However, it is unknown
if the current power standard of 0.8 is reasonable in surgical science.
Methods
PubMed was searched for abstracts published in Surgery, JAMA Surgery, and Annals of
Surgery and from January 1, 2012 to December 31, 2016, with Medical Subject Heading
terms of randomized controlled trial (RCT) or observational study (OBS) and limited
to humans were included (n = 403). Articles were excluded if all reported findings were statistically significant
(n = 193), or if presented data were insufficient to calculate power (n = 141).
Results
A total of 69 manuscripts (59 RCTs and 10 OBSs) were assessed. Overall, the median
power was 0.16 (interquartile range [IQR] 0.08-0.32). The median power was 0.16 for
RCTs (IQR 0.08-0.32) and 0.14 for OBSs (IQR 0.09-0.22). Only 4 studies (5.8%) reached
or exceeded the current 0.8 standard. Two-thirds of our study sample had an a priori
power calculation (n = 41).
Conclusions
High-impact surgical science was routinely unable to reach the arbitrary power standard
of 0.8. The academic surgical community should reconsider the power threshold as it
applies to surgical investigations. We contend that the blueprint for the redesign
should include benchmarking the power of articles on a gradient scale, instead of
aiming for an unreasonable threshold.
Keywords
To read this article in full you will need to make a payment
Purchase one-time access:
Academic & Personal: 24 hour online accessCorporate R&D Professionals: 24 hour online accessOne-time access price info
- For academic or personal research use, select 'Academic and Personal'
- For corporate R&D use, select 'Corporate R&D Professionals'
Subscribe:
Subscribe to Journal of Surgical ResearchAlready a print subscriber? Claim online access
Already an online subscriber? Sign in
Register: Create an account
Institutional Access: Sign in to ScienceDirect
References
- Randomized controlled trials of surgical interventions.Ann Surg. 2010; 251: 409-416
- Negative results of randomized clinical trials published in the surgical literature - equivalency or error?.Arch Surg. 2001; 136: 796-800
- Are surgical trials with negative results being interpreted correctly?.J Am Coll Surg. 2013; 216: 158-166
- The prevalence of negative studies with inadequate statistical power: an analysis of the plastic surgery literature.Plast Reconstr Surg. 2002; 109: 1-6
- Sample size calculations in surgery: are they done correctly?.Surgery. 2003; 134: 275-279
- Handbook of Clinical Psychology.McGraw-Hill, Inc, USA1965
- Statistical Power Analysis for the Behavioral Sciences. 2. Lawrence Earlbaum Associates, Hilsdale, NJ1988
- Demystifying sample-size calculation for clinical trials and comparative effectiveness research: the impact of low-event frequency in surgical clinical research.Surg Endosc. 2013; 27: 359-363
- A proposal to mitigate the consequences of type 2 error in surgical science.Ann Surg. 2018; 267: 621-622
- Post hoc power: a surgeon's first assistant in interpreting “negative” studies.Ann Surg. 2018; 269: e11-e12
- Negative results of randomized clinical trials published in the surgical literature: equivalency or error?.Arch Surg. 2001; 136: 796-800
- Brief report: post hoc power, observed power, a priori power, retrospective power, prospective power, achieved power: sorting out appropriate uses of statistical power analyses.Commun Methods Meas. 2007; 1: 291-299
- Why should surgeons care about clinical research methodology?.J Am Coll Surg. 2006; 203: 827-830
- The Constitution: Amendments 11-27. The United States National Archives and Records Administration,
College Park, MD2016 (Available at)https://www.archives.gov/founding-docs/amendments-11-27Date accessed: April 17, 2019
- The proposal to lower P value thresholds to .005.JAMA. 2018; 319: 1429-1430
- The importance of beta, the type error and sample size in the design and interpreation of the randomized control trial.N Engl J Med. 1978; 299: 690-694
Article info
Publication history
Published online: April 27, 2019
Accepted:
March 27,
2019
Received in revised form:
February 24,
2019
Received:
December 13,
2018
Identification
Copyright
© 2019 Elsevier Inc. All rights reserved.
