Advertisement

Event Rates in Randomized Clinical Trials Evaluating Cardiovascular Interventions and Devices

      Randomized clinical trials (RCTs) are considered the gold standard for evidence-based medicine. However, an accurate estimation of the event rate is crucial for their ability to test clinical hypotheses. Overestimation of event rates reduces the required sample size but can compromise the statistical power of the RCT. Little is known about the prevalence, extent, and impact of overestimation of event rates. The latest RCTs on 10 preselected topics in the field of cardiovascular interventions and devices were selected, and actual primary event rates in the control group were compared with their respective event rate estimations. We also assessed what proportion of the nonsignificant RCTs was truly able to exclude a relevant treatment effect. A total of 27 RCTs randomizing 19,436 patients were included. The primary event rate in the control group was overestimated in 20 of the 27 RCTs (74.1%) resulting in a substantial relative difference between observed and estimated event rates (mean −22.9%, 95% confidence interval −33.5% to −12.2%; median −16.3%, 95% confidence interval −30.3% to −6.5%). Event rates were particularly overestimated in RCTs on biodegradable polymer drug-eluting coronary stents and renal artery stenting. Of the 14 single end point superiority trials with nonsignificant results, only 3 (21.4%) actually resulted in truly negative conclusions. In conclusion, event rates in RCTs evaluating cardiovascular interventions and devices are frequently overestimated. This under-reported phenomenon has fundamental impact on the design of RCTs and can have an adverse impact on the statistical power of these trials to answer important questions about therapeutic strategies.
      To read this article in full you will need to make a payment

      Purchase one-time access:

      Academic & Personal: 24 hour online accessCorporate R&D Professionals: 24 hour online access
      One-time access price info
      • For academic or personal research use, select 'Academic and Personal'
      • For corporate R&D use, select 'Corporate R&D Professionals'

      Subscribe:

      Subscribe to American Journal of Cardiology
      Already a print subscriber? Claim online access
      Already an online subscriber? Sign in
      Institutional Access: Sign in to ScienceDirect

      References

      1. Streptomycin treatment of pulmonary tuberculosis: a medical research council investigation.
        Br Med J. 1948; 2: 769-782
        • Moher D.
        • Hopewell S.
        • Schulz K.F.
        • Montori V.
        • Gotzsche P.C.
        • Devereaux P.J.
        • Elbourne D.
        • Egger M.
        • Altman D.G.
        CONSORT 2010 explanation and elaboration: updated guidelines for reporting parallel group randomised trials.
        BMJ. 2010; 340: c869
        • Schulz K.F.
        • Grimes D.A.
        Sample size calculations in randomised trials: mandatory and mystical.
        Lancet. 2005; 365: 1348-1353
        • Bhatt D.L.
        • Kandzari D.E.
        • O'Neill W.W.
        • D'Agostino R.
        • Flack J.M.
        • Katzen B.T.
        • Leon M.B.
        • Liu M.
        • Mauri L.
        • Negoita M.
        • Cohen S.A.
        • Oparil S.
        • Rocha-Singh K.
        • Townsend R.R.
        • Bakris G.L.
        • SYMPLICITY HTN-3 Investigators
        A controlled trial of renal denervation for resistant hypertension.
        N Engl J Med. 2014; 370: 1393-1401
        • Tahk S.J.
        • Choi B.J.
        • Choi S.Y.
        • Yoon M.H.
        • Gwon H.C.
        • Hong G.R.
        • Kim Y.J.
        • Hur S.H.
        • Kim K.B.
        • Koo B.K.
        • Lee S.H.
        • Yoon J.
        Distal protection device protects microvascular integrity during primary percutaneous intervention in acute myocardial infarction: a prospective, randomized, multicenter trial.
        Int J Cardiol. 2008; 123: 162-168
        • Stone G.W.
        • Webb J.
        • Cox D.A.
        • Brodie B.R.
        • Qureshi M.
        • Kalynych A.
        • Turco M.
        • Schultheiss H.P.
        • Dulas D.
        • Rutherford B.D.
        • Antoniucci D.
        • Krucoff M.W.
        • Gibbons R.J.
        • Jones D.
        • Lansky A.J.
        • Mehran R.
        • Enhanced Myocardial Efficacy and Recovery by Aspiration of Liberated Debris (EMERALD) Investigators
        Distal microcirculatory protection during percutaneous coronary intervention in acute ST-segment elevation myocardial infarction: a randomized controlled trial.
        JAMA. 2005; 293: 1063-1072
        • Holmes Jr., D.R.
        • Kar S.
        • Price M.J.
        • Whisenant B.
        • Sievert H.
        • Doshi S.K.
        • Huber K.
        • Reddy V.Y.
        Prospective randomized evaluation of the Watchman Left Atrial Appendage Closure device in patients with atrial fibrillation versus long-term warfarin therapy: the PREVAIL trial.
        J Am Coll Cardiol. 2014; 64: 1-12
        • Brown M.B.
        • Forsythe A.B.
        Robust tests for the equality of variances.
        J Am Stat Assoc. 1974; 69: 364-367
        • Barbato J.E.
        • Dillavou E.
        • Horowitz M.B.
        • Jovin T.G.
        • Kanal E.
        • David S.
        • Makaroun M.S.
        A randomized trial of carotid artery stenting with and without cerebral protection.
        J Vasc Surg. 2008; 47: 760-765
        • Acker M.A.
        • Parides M.K.
        • Perrault L.P.
        • Moskowitz A.J.
        • Gelijns A.C.
        • Voisine P.
        • Smith P.K.
        • Hung J.W.
        • Blackstone E.H.
        • Puskas J.D.
        • Argenziano M.
        • Gammie J.S.
        • Mack M.
        • Ascheim D.D.
        • Bagiella E.
        • Moquete E.G.
        • Ferguson T.B.
        • Horvath K.A.
        • Geller N.L.
        • Miller M.A.
        • Woo Y.J.
        • D'Alessandro D.A.
        • Ailawadi G.
        • Dagenais F.
        • Gardner T.J.
        • O'Gara P.T.
        • Michler R.E.
        • Kron I.L.
        • CTSN
        Mitral-valve repair versus replacement for severe ischemic mitral regurgitation.
        N Engl J Med. 2014; 370: 23-32
        • Smits P.C.
        • Hofma S.
        • Togni M.
        • Vazquez N.
        • Valdes M.
        • Voudris V.
        • Slagboom T.
        • Goy J.J.
        • Vuillomenet A.
        • Serra A.
        • Nouche R.T.
        • den Heijer P.
        • van der Ent M.
        Albuminal biodegradable polymer biolimus-eluting stent versus durable polymer everolimus-eluting stent (COMPARE II): a randomised, controlled, non-inferiority trial.
        Lancet. 2013; 381: 651-660
        • Meier B.
        • Kalesan B.
        • Mattle H.P.
        • Khattab A.A.
        • Hildick-Smith D.
        • Dudek D.
        • Andersen G.
        • Ibrahim R.
        • Schuler G.
        • Walton A.S.
        • Wahl A.
        • Windecker S.
        • Juni P.
        • PC Trial Investigators
        Percutaneous closure of patent foramen ovale in cryptogenic embolism.
        N Engl J Med. 2013; 368: 1083-1091
        • Cooper C.J.
        • Murphy T.P.
        • Cutlip D.E.
        • Jamerson K.
        • Henrich W.
        • Reid D.M.
        • Cohen D.J.
        • Matsumoto A.H.
        • Steffes M.
        • Jaff M.R.
        • Prince M.R.
        • Lewis E.F.
        • Tuttle K.R.
        • Shapiro J.I.
        • Rundback J.H.
        • Massaro J.M.
        • D'Agostino R.B.S.
        • Dworkin L.D.
        • CORAL Investigators
        Stenting and medical therapy for atherosclerotic renal-artery stenosis.
        N Engl J Med. 2014; 370: 13-22
        • Wheatley K.
        • Ives N.
        • Gray R.
        • Kalra P.A.
        • Moss J.G.
        • Baigent C.
        • Carr S.
        • Chalmers N.
        • Eadington D.
        • Hamilton G.
        • Lipkin G.
        • Nicholson A.
        • Scoble J.
        • ASTRAL Investigators
        Revascularization versus medical therapy for renal-artery stenosis.
        N Engl J Med. 2009; 361: 1953-1962
        • Bax L.
        • Woittiez A.J.
        • Kouwenberg H.J.
        • Mali W.P.
        • Buskens E.
        • Beek F.J.
        • Braam B.
        • Huysmans F.T.
        • Schultze Kool L.J.
        • Rutten M.J.
        • Doorenbos C.J.
        • Aarts J.C.
        • Rabelink T.J.
        • Plouin P.F.
        • Raynaud A.
        • van Montfrans G.A.
        • Reekers J.A.
        • van den Meiracker A.H.
        • Pattynama P.M.
        • van de Ven P.J.
        • Vroegindeweij D.
        • Kroon A.A.
        • de Haan M.W.
        • Postma C.T.
        • Beutler J.J.
        Stent placement in patients with atherosclerotic renal artery stenosis and impaired renal function: a randomized trial.
        Ann Intern Med. 2009; 150: 840-848
        • Holmes D.R.
        • Reddy V.Y.
        • Turi Z.G.
        • Doshi S.K.
        • Sievert H.
        • Buchbinder M.
        • Mullin C.M.
        • Sick P.
        • PROTECT AF Investigators
        Percutaneous closure of the left atrial appendage versus warfarin therapy for prevention of stroke in patients with atrial fibrillation: a randomised non-inferiority trial.
        Lancet. 2009; 374: 534-542
        • Carroll J.D.
        • Saver J.L.
        • Thaler D.E.
        • Smalling R.W.
        • Berry S.
        • MacDonald L.A.
        • Marks D.S.
        • Tirschwell D.L.
        • RESPECT Investigators
        Closure of patent foramen ovale versus medical therapy after cryptogenic stroke.
        N Engl J Med. 2013; 368: 1092-1100
        • Nielsen H.H.
        • Klaaborg K.E.
        • Nissen H.
        • Terp K.
        • Mortensen P.E.
        • Kjeldsen B.J.
        • Jakobsen C.J.
        • Andersen H.R.
        • Egeblad H.
        • Krusell L.R.
        • Thuesen L.
        • Hjortdal V.E.
        A prospective, randomised trial of transapical transcatheter aortic valve implantation vs. surgical aortic valve replacement in operable elderly patients with aortic stenosis: the STACCATO trial.
        EuroIntervention. 2012; 8: 383-389
        • Weaver C.S.
        • Leonardi-Bee J.
        • Bath-Hextall F.J.
        • Bath P.M.
        Sample size calculations in acute stroke trials: a systematic review of their reporting, characteristics, and relationship with outcome.
        Stroke. 2004; 35: 1216-1224
        • Charles P.
        • Giraudeau B.
        • Dechartres A.
        • Baron G.
        • Ravaud P.
        Reporting of sample size calculation in randomised controlled trials: review.
        BMJ. 2009; 338: b1732
        • Goldman A.I.
        • Holcomb R.
        • Perry Jr., H.M.
        • Schnaper H.W.
        • Fitz A.E.
        • Frohlich E.D.
        Can dropout and other noncompliance be minimized in a clinical trial? Report from the Veterans Administrative National Heart, Lung and Blood Institute cooperative study on antihypertensive therapy: mild hypertension.
        Control Clin Trials. 1982; 3: 75-89
        • Ferrie J.E.
        • Kivimaki M.
        • Singh-Manoux A.
        • Shortt A.
        • Martikainen P.
        • Head J.
        • Marmot M.
        • Gimeno D.
        • De Vogli R.
        • Elovainio M.
        • Shipley M.J.
        Non-response to baseline, non-response to follow-up and mortality in the Whitehall II cohort.
        Int J Epidemiol. 2009; 38: 831-837
        • Sacks H.
        • Chalmers T.C.
        • Smith Jr., H.
        Randomized versus historical controls for clinical trials.
        Am J Med. 1982; 72: 233-240
        • Benson K.
        • Hartz A.J.
        A comparison of observational studies and randomized, controlled trials.
        N Engl J Med. 2000; 342: 1878-1886
        • de Boer S.P.
        • Lenzen M.J.
        • Oemrawsingh R.M.
        • Simsek C.
        • Duckers H.J.
        • van der Giessen W.J.
        • Serruys P.W.
        • Boersma E.
        Evaluating the ‘all-comers’ design: a comparison of participants in two ‘all-comers’ PCI trials with non-participants.
        Eur Heart J. 2011; 32: 2161-2167
        • Roethlisberger F.J.
        • Dickson W.J.
        • Wright H.A.
        • Western Electric Company
        Management and the Worker: An Account of a Research Program Conducted by the Western Electric Company, Hawthorne Works, Chicago.
        Harvard University Press, Cambridge1939: 615
        • Brinker S.
        • Pandey A.
        • Ayers C.
        • Price A.
        • Raheja P.
        • Arbique D.
        • Das S.R.
        • Halm E.A.
        • Kaplan N.M.
        • Vongpatanasin W.
        Therapeutic drug monitoring facilitates blood pressure control in resistant hypertension.
        J Am Coll Cardiol. 2014; 63: 834-835
        • Head S.J.
        • Kaul S.
        • Bogers A.J.
        • Kappetein A.P.
        Non-inferiority study design: lessons to be learned from cardiovascular trials.
        Eur Heart J. 2012; 33: 1318-1324
        • Gould A.L.
        Planning and revising the sample size for a trial.
        Stat Med. 1995; 14: 1039-1051
        • Halpern S.D.
        • Karlawish J.H.
        • Berlin J.A.
        The continuing unethical conduct of underpowered clinical trials.
        JAMA. 2002; 288: 358-362