Rosiglitazone: can meta-analysis accurately estimate excess cardiovascular risk given the available data? Re-analysis of randomized trials using various methodologic approaches
© Friedrich et al; licensee BioMed Central Ltd. 2009
Received: 20 October 2008
Accepted: 10 January 2009
Published: 10 January 2009
A recent and provocative meta-analysis, based on few outcome events, suggested that rosiglitazone increased cardiovascular mortality and myocardial infarction. However, results of meta-analyses of trials with sparse events, often performed when examining uncommon adverse effects due to common therapies, can vary substantially depending on methodologic decisions. The objective of this study was to assess the robustness of the rosiglitazone results by using alternative reasonable methodologic approaches and by analyzing additional related outcomes.
In duplicate and independently, we abstracted all myocardial and cerebrovascular ischemic events from all randomized controlled trials listed on the manufacturer's web site meeting inclusion criteria of the original meta-analysis (at least 24 weeks of rosiglitazone exposure in the intervention group and any control group without rosiglitazone). We performed meta-analyses of these data under different methodologic conditions. An unconfounded comparison that includes only trials (or arms of trials) in which medications apart from rosiglitazone are identical suggests higher risks than previously reported, making even the risk of cardiovascular death statistically significant. Alternatively, meta-analysis that includes all trials comparing a treatment arm receiving rosiglitazone to any control arm without rosiglitazone (as in the original meta-analysis) but also including trials with no events in both the rosiglitazone and control arms (not incorporated in the original meta-analysis), shows adverse but non-statistically significant effects of rosiglitazone on myocardial infarction and cardiovascular mortality. Rosiglitazone appears to have inconsistent effects on a wider range of cardiovascular outcomes. It increases the risk of a broad range of myocardial ischemic events (not just myocardial infarction). However, its effect on cerebrovascular ischemic events suggests benefit, although far from statistically significant.
We have shown that alternative reasonable methodological approaches to the rosiglitazone meta-analysis can yield increased or decreased risks that are either statistically significant or not significant at the p = 0.05 level for both myocardial infarction and cardiovascular death. Completion of ongoing trials may help to generate more accurate estimates of rosiglitazone's effect on cardiovascular outcomes. However, given that almost all point estimates suggest harm rather than benefit and the availability of alternative agents, the use of rosiglitazone may greatly decline prior to more definitive safety data being generated.
A recent  and provocative [2–4] meta-analysis suggested that rosiglitazone increased cardiovascular morbidity and mortality. Given the popularity of this medication, the elevated risks of myocardial infarction (MI) (Peto odds ratio [OR] 1.43; 95% confidence interval [CI] 1.03–1.98; p = 0.03) and cardiovascular death (OR 1.64; 95% CI 0.98–2.74; p = 0.06) had broad public health implications. When interim results are added from the subsequently published RECORD trial , a phase III trial investigating cardiovascular outcomes, the effect on cardiovascular death is not significant (OR 1.08; 95% CI 0.78–1.51; p = 0.64) but the increased risk of MI remains (OR 1.35; 95% CI 1.04–1.75; p = 0.02). Other meta-analyses conducted by rosiglitazone's manufacturer , the US Food and Drug Administration , and the Cochrane Collaboration , in addition to two additional independent meta-analyses [9, 10], supported these findings .
Meta-analysts  and others  have highlighted limitations, such as the short duration of many included trials that were not designed to assess cardiovascular outcomes, leading to potential ascertainment bias, in addition to sparse outcome events . Indeed, results of a meta-analysis of trials with sparse events can vary substantially depending on methodologic and statistical decisions [11–13]. The objective of this study was to assess the robustness of the rosiglitazone results using alternative reasonable methodologic approaches and analyzing additional related outcomes. In particular, we examine four relevant issues not evaluated in previous analyses: 1) the effect of including only trials (or arms of trials) with an unconfounded comparison, in which medications apart from rosiglitazone are identical in the arms being compared, 2) the effect of using odds ratios, which can be biased when events are infrequent and group sizes are imbalanced, 3) the effect of including trials with no events in both groups, which would reduce pooled estimates of rosglitazone's effect, and 4) the effect of various definitions of ischemia outcomes (to examine for consistency of effect).
All randomized controlled trials on the manufacturer's website  meeting inclusion criteria (any control group and at least 24 weeks of drug exposure) for the original meta-analysis  were reviewed. Our aim was to abstract data from the same database of clinical trials searched in the original meta-analysis , in addition to adding interim data from the RECORD trial published in response to the original meta-analysis. In duplicate, we selected trials from the database and abstracted data from included trials.
From each included trial, we abstracted data (number of patients in intervention and control arm with an outcome event and total patients in each arm) on the outcomes of myocardial infarction and cardiovascular death. For the outcome of myocardial ischemia, we included all the following events (each was reported in at least one trial): myocardial infarction (154 patients), myocardial ischemia (301 patients), new angina (44 patients), angina (36 patients), aggravated angina (19 patients), unstable angina (8 patients), cardiac chest pain (1 patient), coronary artery insufficiency (2 patients), revascularization (62 patients), coronary thrombosis (6 patients), coronary artery stenosis (2 patients), coronary artery occlusion (2 patients), coronary artery disorder (15 patients), coronary artery disease (7 patients), coronary artery atherosclerosis (2 patients), coronary artery spasm (1 patient), and three vessel disease (1 patient). For the outcome of cerebrovascular event we included all the following events (each was reported in at least one trial): stroke (64 patients), cerebrovascular accident (5 patients), cerebral infarction (1 patient), cerebral embolism (1 patient), cerebrovascular disorder (43 patients), transient ischemic attack (31 patients), cerebral ischemia (1 patient), vestibulobasilar insufficiency (1 patient), and carotid stenosis (2 patients). For both myocardial ischemia and cerebrovascular events, trials did not specify which patients had more than one event among those listed above. Therefore, if a trial reported outcome events in more than one category, in our primary analysis we assumed that the number of patients with an event was equal to the maximum number of patients in any one category. In a sensitivity analysis we used the least conservative approach and assumed that each adverse event occurred in a different patient; our results did not change.
Interim data for RECORD, which was not included in the manufacturer's website , was obtained from reference 5 (myocardial infarction and cardiovascular death) and reference 7 (stroke: 29 strokes in 2220 rosiglitazone patients and 38 strokes in 2227 control patients). Data on other myocardial and cerebrovascular ischemic events were not available for RECORD.
Binary effect measure meta-analyses were carried out using standard equations and confirmed with Review Manager 4.2 (Cochrane Collaboration, Oxford, England) where possible. Odds ratios derived using exact statistical methods were calculated with StatXact 8 (Cytel Inc, Cambridge, MA, USA) and odds ratios derived using Bayesian methods were calculated with WinBUGS 1.4.1 [available at http://www.mrc-bsu.cam.ac.uk/bugs/]).
Influence of Trial Selection Criteria
Influence of Effect Measure
Influence of the Exclusion of Trials with No Outcome Events
Regardless of the specific form of odds ratio, however, this effect measure forces the exclusion of studies with zero events in both the intervention and control arms . This occurs because odds ratios from such studies become undefined due to division by zero when standard methods are used . In addition to the 42 trials in the original meta-analysis , we identified an additional 12 trials on the manufacturer's web site  meeting inclusion criteria but with no myocardial infarctions (though some listed other myocardial ischemic or cerebrovascular events) and no deaths from cardiovascular causes in both intervention and control arms. Evaluating all 54 trials, 16 trials with 4241 patients (30% of all trials; 14% of 30242 patients in all trials) recorded no myocardial infarctions and 31 trials with 9801 patients (57% of trials; 32% of 30242 patients) recorded no cardiovascular deaths.
Consistency of Effect across Related Outcomes and over Time
Clinicians and regulators grappling with the implications of important but subtle distinctions in meta-analysis methodology may look to corollary evidence to guide decisions regarding rosiglitazone. One approach to examine the robustness and consistency of the myocardial infarction results would be to analyze rosiglitazone's effect on all events resulting from myocardial ischemia. Doing so and using the same statistical methods also suggests increased risk (Figure 3). A related approach would be to examine whether rosiglitazone's effect is similar in different vascular territories. When the same methods are used for cerebrovascular ischemic events, there is no evidence of harm and, in fact, some suggestion of benefit (Figure 3). Thus, there appears to be consistency of effect using various definitions of cardiac morbidity, but a lack of consistency between cardiac and cerebrovascular effects. This difference may reflect unknown mechanisms of rosiglitazone's effects on different vascular territories (for example, increased congestive heart failure may contribute to increased myocardial ischemia with little effect on cerebral ischemia) or may simply highlight chance differences in trial outcomes when absolute event rates are low.
Inferences regarding rosiglitazone's effect on adverse cardiovascular events, based on meta-analysis of available randomized controlled trials studying primarily low-risk patients for relatively short intervals, depend on selection of trials and effect measure. We have demonstrated that different reasonable methodological approaches can yield increased or decreased risks that are either statistically significant or not significant at the p = 0.05 level for both MI and cardiovascular death. Overall, our new analyses support concerns regarding increased cardiovascular risks of rosiglitazone. However, they also highlight the challenge of attributing rare adverse events to a particular drug and the importance of using sensitivity analyses when results are of borderline statistical significance.
We examined four methodologic issues relevant to a meta-analysis of rosiglitazone's effects that were not completely considered in previous meta-analyses [1, 2, 6–10]. First, we determined the overall unconfounded effect of rosiglitazone by comparing groups differing only by rosiglitazone therapy and showed higher estimates of harm that achieve statistical significance even for cardiovascular death. Second, we used other less biased effect measures for low event rate data and found similar effects. Third, we explored consistency of effects across outcomes; using a broader definition of myocardial ischemia events we also found statistically significant harm, but we did not find an increased risk of cerebrovascular events. Finally, we included additional trials with no outcome events, an approach which decreases the magnitude of treatment effects, and found that all risk difference point estimates for rosiglitazone still suggest harm rather than benefit, although some p-values are not statistically significant. Others  have also re-calculated inverse variance and Mantel-Haenszel odds ratios by including zero total event trials (using methods suggested by us ) and showed reduced estimates of harm due to rosiglitazone. However, their results  are likely strongly influenced by the bias towards no effect exhibited by inverse variance and Mantel-Haenszel odds ratios when event rates are very low. In contrast, our analyses including zero total event trials used an effect measure that minimized this bias to evaluate the impact of such trials for rosiglitazone.
In summary, our additional new analyses strengthen the original results, which have been questioned because of sparse outcome events and borderline statistical significance . Ideally, a large randomized trial adequately powered to detect differences in infrequent cardiovascular adverse events would be conducted, especially since results from such trials often differ from meta-analyses of smaller trials . However, given the very small absolute increase in event rates, such a trial would need to be extremely large. Ongoing  and future clinical trials may help to generate more accurate estimates of rosiglitazone's effect on cardiovascular outcomes, but will need to study higher-risk patients for longer time periods, as rosiglitazone is intended for life-long use. Unfortunately, ongoing trials like RECORD may still be underpowered to provide definitive data due to lower-than expected events rates and other trial design issues .
Other related data support concerns about rosiglitazone's effects. A population-based observational study , despite also having relatively few events and a study design more prone to confounding, also suggests increased cardiovascular risk from rosiglitazone. In addition, another recent meta-analysis  without the same statistical issues suggests that pioglitazone, the other commercially available thiazolidinedione, is not associated with increased cardiovascular morbidity. The statistical issues discussed here are less important for the pioglitazone meta-analysis because the event rates are higher and the proportion of zero event rate trials is probably lower (the exact proportion of such trials, considering both included and excluded trials in the denominator, cannot be calculated from data available in the paper ). Moreover, the largest trial, a phase III trial, with a mean follow up of almost 3 years , which compared pioglitazone to placebo, dominated the results. This trial almost demonstrated a statistically significant benefit in the primary endpoint that included all-cause mortality, non-fatal myocardial infarction, and stroke, among others, on its own. Thus, data from these related studies makes an adequately powered trial simply to disprove harm difficult to justify from both ethical (potential harm to patients) and resource perspectives.
Given the growing population with type 2 diabetes and their high prevalence of cardiovascular disease, the accurate and reliable determination of any cardiovascular risks and benefits has enormous public health implications. However, because available clinical trial data for rosiglitazone do not suggest improved clinical outcomes and possible harm, and because alternative agents are available, clinicians may simply stop prescribing rosiglitazone without definitive safety conclusions ever being generated. Although such a situation is not ideal, given the available imperfect data, such a course of action does not appear unreasonable.
By Sonal Singh1 and Yoon K. Loke2
Address: 1Wake Forest University School of Medicine, Winston-Salem, NC 27157, USA.
2University of East Anglia, Norwich NR4 7TJ, UK.
This report by Friedrich et al. utilizes various meta-analytic techniques to measure the cardiovascular risk of rosiglitazone. Irrespective of the methodology used, the results are the same. They report a similar increased risk of myocardial infarction (and cardiovascular death-statistically significant at times) as seen in our rigorous meta-analysis of long term trials with rosiglitazone, which showed that rosiglitazone is associated with increased ischemic risk with long term use in patients with type 2 diabetes as compared to alternative therapies .
However, there are several challenges in measuring rare but important adverse effects in randomized trials because of incomplete reporting of outcomes and inconsistent definitions. The short term trials may be inadequately powered to detect long term adverse effects, and hence we focused on the long term trials. We used the latest version of the published data, and consistently extracted similar categories of events such as those requiring hospitalization or classified as serious adverse events, irrespective of whether adjudicated or not because different trials may have different ways of adjudication . The use of adjudicated and non-adjudicated events may give different risk estimates. We also provided sensitivity estimates based on the adjudicated events which showed a similar increased risk .
The inclusion of zero event trials as done by the authors of this article is problematic as it is unclear if no events were reported or whether events did not occur. Several of these problems can be fixed by consistent reporting of outcomes of all trials and methodological consensus on how best to conduct meta-analysis on sparse events.
However the authors overemphasize the precise estimate of the RRs and the corresponding P values, which are likely to vary given the different inclusion and exclusion criteria. More important than the absolute precisions of risk, the consistency of effects across comparisons suggests a higher cardiovascular risk with rosiglitazone compared to placebo or active controls, despite the modest benefit in glycemic control seen with rosiglitazone. Finally observational studies have also proved to be useful and shown similar increased cardiovascular risk with rosiglitazone [21, 24].
Hence accurate estimates of cardiac risk on ongoing trials are unlikely to change this general scientific consensus recommendations by the American Diabetes Association that " rosiglitazone be avoided in the treatment of type 2 diabetes . Infact, one can argue that it would be unethical to conduct such a trial, and expose patients to harm, to measure the precise cardiovascular risk of rosiglitazone, given the overwhelming existing evidence on the cardiovascular adverse effects of rosiglitazone [26, 27]. Unfortunately a drug used to treat type 2 diabetes, where 2/3rd of deaths are due to cardiovascular death, should at the minimum be cardiac neutral, which is not the case with rosiglitazone which raises the risk of cardiovascular adverse effects no matter what methodology is used.
Competing interests: None
The authors thank John A. Tayek for helpful comments, and Ruxandra Pinto for assistance with some of the figures. This study received no specific funding. Jan Friedrich is supported by a Canadian Institutes of Health Research Clinician-Scientist Award. The Canadian Institutes of Health Research had no involvement in the conduct of this study.
- Nissen SE, Wolski K: Effect of rosiglitazone on the risk of myocardial infarction and death from cardiovascular causes. N Engl J Med. 2007, 356: 2457-2471. 10.1056/NEJMoa072761. [Erratum. N Engl J Med 2007, 357: 100.]View ArticlePubMedGoogle Scholar
- Diamond GA, Bax L, Kaul S: Uncertain effects of rosiglitazone on the risk of myocardial infarction and cardiovascular death. Ann Intern Med. 2007, 147: 578-581.View ArticlePubMedGoogle Scholar
- Anonymous: Rosiglitazone: seeking a balanced perspective. Lancet. 2007, 369 (9576): 1834-Google Scholar
- Rosen CJ: The rosiglitazone story – lessons from an FDA advisory committee meeting. N Engl J Med. 2007, 357: 844-846. 10.1056/NEJMp078167.View ArticlePubMedGoogle Scholar
- Home PD, Pocock SJ, Beck-Nielsen H, Gomis R, Hanefeld M, Jones NP, Komajda M, McMurray JJV, for the RECORD Study Group: Rosiglitazone evaluated for cardiovascular outcomes – an interim analysis. N Engl J Med. 2007, 357: 28-38. 10.1056/NEJMoa073394.View ArticlePubMedGoogle Scholar
- Richter B, Bandiera-Echtler E, Bergerhoff K, Clar C, Ebrahim S: Rosiglitazone for type 2 diabetes mellitus. Cochrane Database Syst Rev. 2007, 3: CD006063-PubMedGoogle Scholar
- Division of Metabolism and Endocrine Products and Office of Surveillance and Epidemiology, US Food and Drug Administration: FDA Briefing Document: joint meeting of the Endocrinologic and Metabolic Drugs Advisory Committee. July 30, 2007. http://www.fda.gov/ohrms/dockets/ac/07/briefing/2007-4308b1-02-fda-backgrounder.pdf and http://www.fda.gov/ohrms/dockets/ac/07/briefing/2007-4308b1-01-sponsor-backgrounder.pdf
- GlaxoSmithKline clinical trial register: Study No. ZM2005/00181/01, Avandia Cardiovascular Event Modeling Project, and study No. HM2006/00497/00WEUSRTP866, Coronary Heart Disease Outcomes in Patients Receiving Antidiabetic Agents. [http://ctr.gsk.co.uk/Summary/Rosiglitazone/III_CVmodeling.pdf]
- Singh S, Loke YK, Furberg CD: Long-term risk of cardiovascular events with rosiglitazone. A meta-analysis. JAMA. 2007, 298: 1189-1195. 10.1001/jama.298.10.1189.View ArticlePubMedGoogle Scholar
- Lago RM, Singh PP, Nesto RW: Congestive heart failure and cardiovascular death in patients with prediabetes and type 2 diabetes given thiazolidinediones: a meta-analysis of randomised clinical trials. Lancet. 2007, 370: 1129-1136. 10.1016/S0140-6736(07)61514-1.View ArticlePubMedGoogle Scholar
- Sweeting MJ, Sutton AJ, Lambert PC: What to add to nothing? Use and avoidance of continuity corrections in meta-analysis of sparse data. Stat Med. 2004, 23: 1351-1375. 10.1002/sim.1761. [Erratum. Stat Med 2006, 25: 2700.]View ArticlePubMedGoogle Scholar
- Bradburn MJ, Deeks JJ, Berlin JA, Russell Localio A: Much ado about nothing: a comparison of the performance of meta-analytical methods with rare events. Stat Med. 2007, 26: 53-77. 10.1002/sim.2528.View ArticlePubMedGoogle Scholar
- Friedrich JO, Adhikari NKJ, Beyene J: Inclusion of zero total event trials in meta-analyses maintains analytic consistency and incorporates all available data. BMC Med Res Methodol. 2007, 7: 5-10.1186/1471-2288-7-5.PubMed CentralView ArticlePubMedGoogle Scholar
- GlaxoSmithKline trial register. , [http://ctr.gsk.co.uk/Summary/rosiglitazone/studylist.asp]
- Bracken MB: Rosiglitazone and cardiovascular risk. N Engl J Med. 2007, 357: 937-938. 10.1056/NEJMc071602.View ArticlePubMedGoogle Scholar
- Diamond GA, Kaul S: Rosiglitazone and cardiovascular risk. N Engl J Med. 2007, 357: 938-939.PubMedGoogle Scholar
- Kahn SE, Haffner SM, Heise MA, Herman WH, Holman RR, Jones NP, Kravitz BG, Lachin JM, O'Neill MC, Zinman B, Viberti G, for the ADOPT Study Group: Glycemic durability of rosiglitazone, metformin, or glyburide monotherapy. N Engl J Med. 2006, 355: 2427-2443. 10.1056/NEJMoa066224.View ArticlePubMedGoogle Scholar
- Gerstein HC, Yusuf S, Holman RR, Bosch J, Diabetes Reduction Assessment With Ramipril and Rosiglitazone Medication Trial Investigators: Effect of rosiglitazone on the frequency of diabetes in patients with impaired glucose tolerance or impaired fasting glucose: a randomized controlled trial. Lancet. 2006, 368: 1096-1105. 10.1016/S0140-6736(06)69829-2.View ArticlePubMedGoogle Scholar
- Dargie HJ, Hilderbrandt PR, Riegger GA, McMurray JJV, McMorn SO, Roberts JN, Zambanini A, Wilding JPH: A randomized, placbo-controlled trial assessing the effects of rosiglitazone on echocardiographic function and cardiac status in type 2 diabetic patients with New York Heart Association functional class I or II heart failure. J Am Coll Cardiol. 2007, 49: 1696-1704. 10.1016/j.jacc.2006.10.077. Also GlaxoSmithKline clinical trial register: study no. 49653/211., [http://ctr.gsk.co.uk/Summary/rosiglitazone/IV_49653_211.pdf]View ArticlePubMedGoogle Scholar
- Cappelleri JC, Ioannidis JPA, Schmid CH, de Ferranti SD, Aubert M, Chalmers TC, Lau J: Large trials versus meta-analyses of smaller trials: How do their results compare?. JAMA. 1996, 276: 1332-1338. 10.1001/jama.276.16.1332.View ArticlePubMedGoogle Scholar
- Lipscombe LL, Gomes T, Levesque LE, Hux JE, Juurlink DN, Alter DA: Thiazolidinediones and cardiovascular outcomes in older patients with diabetes. JAMA. 2007, 298: 2634-2643. 10.1001/jama.298.22.2634.View ArticlePubMedGoogle Scholar
- Lincoff AM, Wolski K, Nicholls SJ, Nissen SE: Pioglitazone and risk of cardiovascular events in patients with type 2 diabetes mellitus. A meta-analysis of randomized trials. JAMA. 2007, 298: 1180-1188. 10.1001/jama.298.10.1180.View ArticlePubMedGoogle Scholar
- Dormandy JA, Charbonnel B, Eckland DJA, Erdmann E, Massi-Benedetti M, Moules IK, Skene AM, Tan MH, Lefebvre PJ, Murray GD, Standl E, Wilcox RG, Wilhelmsen L, Betteridge J, Birkeland K, Golay A, Heine RJ, Koranyi L, Laakso M, Mokan M, Norkus A, Pirags V, Podar T, Scheen A, Scherbaum W, Schernthaner G, Schmitz O, Skrha J, Smith U, Taton J, on behalf of the PROactive investigators: Secondary prevention of macrovascular events in patients with type 2 diabetes in the PROactive Study (PROspective pioglitAzone Clinical Trial In macroVascular Events): a randomised controlled trial. Lancet. 2005, 366: 1279-1289. 10.1016/S0140-6736(05)67528-9.View ArticlePubMedGoogle Scholar
- Winkelmayer WC, Setoguchi S, Levin R, Solomon DH: Comparison of cardiovascular outcomes in elderly patients with diabetes who initiated rosiglitazone vs pioglitazone therapy. Arch Intern Med. 2008, 168: 2368-2375. 10.1001/archinte.168.21.2368.View ArticlePubMedGoogle Scholar
- Nathan DM, Buse JB, Davidson MB, Ferrannini E, Holman RR, Sherwin R, Zinman B: Medical Management of Hyperglycemia in Type 2 Diabetes: a Consensus Algorithm for the Initiation and Adjustment of Therapy. Diabetes Care. 2009, 32: 193-203. 10.2337/dc08-9025.PubMed CentralView ArticlePubMedGoogle Scholar
- Singh S, Furberg CD: Thiazolidinediones and cardiovascular outcomes in type 2 diabetes. Heart. 2009, 95: 1-3. 10.1136/hrt.2008.155507.View ArticlePubMedGoogle Scholar
- Singh S, Loke YK, Furberg CD: Thiazolidinediones and heart failure: a teleo-analysis. Diabetes Care. 2007, 30: 2148-2153. 10.2337/dc07-0141.View ArticlePubMedGoogle Scholar
This article is published under license to BioMed Central Ltd. This is an open access article distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/2.0), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.