doi: 10.1161/CIRCULATIONAHA.107.746917
(Circulation. 2008;117:333-335.)
© 2008 American Heart Association, Inc.
Editorial |
On the Evaluation of Intervention Outcome Risks for Patients With Ischemic Heart Disease
From the Department of Cardiovascular Sciences, Brody School of Medicine at ECU, Greenville, NC.
Correspondence to T. Bruce Ferguson, Jr, MD, Department of Cardiovascular Sciences, East Caroline Heart Institute, 600 Moye Blvd, TA 255, Greenville, NC 27834. E-mail fergusont{at}ecu.edu
Key Words: Editorials • revascularization • risk factors • outcomes research
Risk is defined in the Oxford Pocket American Dictionary of Current English as "a chance or possibility of danger, loss, injury, etc."1 The evaluation of clinical risk and benefit is a mainstay of all clinical investigation in medicine. More recently, it is slowly becoming a component of everyday clinical care. In this issue of Circulation, Singh et al2 potentially advance the evaluation of risk in the domain of ischemic heart disease.
Article p 356
Because procedural mortality has been such an important outcome in the surgical revascularization of coronary artery disease, the assessment of preoperative risk has been an important focus of attention for many years. Large US regional and national databases, such as the Northern New England Cardiovascular Disease Study Group,3 the New York State cardiac surgery database,4 and the Society of Thoracic Surgeons’ National Cardiac Database (STS NCD),5 have developed relatively sophisticated multivariable models to predict mortality risk across the entire spectrum of coronary artery bypass grafting (CABG) patients. Similar risk models have been developed outside the United States, most notably from the European Cardiac Surgery database.6
Model performance has been tested and continually refined within these databases. Over the years, it has become clear that a core group of preoperative risk factors and conditions contribute a majority of the inherent mortality risk in these models. This has led to the concept that specific risk models can be applied to other CABG data sets. Nashef et al7 tested the performance of the EuroSCORE model against a cohort of STS CABG patients. In general, these risk-prediction models achieve the best fit when applied to the database systems from which they were derived.
In the United States, the STS NCD has documented a substantial increase in the predicted risk of mortality over more than a decade for patients coming to CABG, an increase that is due to significant increases in preoperative risk factors.8 Despite this increase in predicted risk, operative mortality for surgical revascularization has undergone a sustained decline during this same time interval, such that the net benefit of CABG today is substantially greater than it was even 5 years ago. Indeed, the overall observed mortality rate for CABG in the STS NCD for 2006 approached 2%, and for elective procedures it was 
1%.9
More recently, insurers and patients have requested that preoperative risk be documented for individual patients before CABG and other adult cardiac surgical procedures. This can now be done by accessing the EuroSCORE10 and STS11 online risk calculators. These online programs allow for entry of preoperative variables and the calculation of a mortality risk score based on the latest version of the database model. The STS Web site also allows for calculation of major comorbidities for CABG patients. However, in the context of care delivery, this risk calculation almost always takes place after the decision for surgical intervention has been made and the patient has been referred to a cardiothoracic surgeon.
Since the early part of this decade, percutaneous coronary intervention (PCI) investigators have developed similar in-hospital mortality risk models, again at regional12 and national13 levels. Probably because mortality in PCI is an uncommon occurrence, it is not clear how frequently these risk-prediction models are used in daily practice. In aggregate, however, these advances in risk prediction in cardiovascular disease have been distinctive and have not been replicated in other areas of medicine.
Singh and colleagues2 have advanced the evaluation of risk a bit further. They applied a predictive risk model for in-hospital mortality developed for PCI revascularization at the Mayo Clinic (the Mayo Clinic Risk Score [MCRS]) to a contemporary cohort of CABG patients from the STS database. The first feature of this study is that the MCRS is designed to be a bedside tool. Calculation of risk by use of bedside tools is not a new development. Almost 20 years ago, Parsonnet et al14 published a bedside CABG mortality risk calculator; this tool performed relatively well in low-risk patients but was highly variable and mostly inaccurate in higher-risk patients. The Northern New England Study Group developed a Northern New England Study database–specific template that allowed for the bedside calculation of CABG mortality risk in their patients.3 Concern about application outside of the development and testing populations, however, has limited widespread adoption of these tools; rather, in this context, they have been used as general guidelines for CABG mortality risk. Recently, bedside tools for evaluation of the risk of complications such as renal failure15 have been published for CABG patients.
The second and more provocative feature of the present study is that it is a seminal application of a risk model across different therapeutic options for the same underlying disease process. The authors tested the performance of the MCRS model in this CABG cohort and compared its ability to predict risk against risk values generated from the STS database model. This PCI risk model was based on 7 preprocedure variables (age, creatinine, ejection fraction, myocardial infarction 
24 hours, shock, congestive heart failure, and peripheral vascular disease). Although the current STS CABG model includes 26 preoperative variables that contribute to mortality, the MCRS model performed reasonably well in evaluating the predicted risk of patients undergoing bypass surgery. As expected, the discriminatory ability of the MCRS model was moderate in CABG patients (C-statistic=0.715 to 0.284 among various subgroups) and was generally inferior to the STS model.
If this bedside tool is somewhat less accurate than the risk calculation available through the STS Web site, what then is its utility? In the context of treatment options for multivessel coronary artery disease, this ability to evaluate preprocedure risk has important implications. In contrast to previous applications of risk-assessment tools, which are principally used after the therapeutic decision for PCI or CABG has been made, a model that could be equitably applied to 2 different forms of therapy to assess relative outcomes makes this risk evaluation an important part of the therapeutic decision-making process. The potential utility of such a model, in short, is that it could be applied before a therapeutic decision for multivessel PCI versus CABG has been made. Use of such an approach would introduce important new information into the context of a multidisciplinary informed consent process for treating multivessel coronary artery disease and to the accurate assessment of the appropriateness of therapy in individual patients. To date, the authors have not tested this proposition, but the results of the present study suggest that it would be feasible, given the performance of this bedside risk model in the PCI domain and the CABG domain.
Additional implications present themselves. First, testing and refinement of such an approach, perhaps with a cohort from the American College of Cardiology’s National Cardiovascular Data Registry and the STS NCD in combination, would solidify the premise that outcomes-based risk assessment can be incorporated into the therapeutic decision-making process. Second, although this risk-prediction model addressed in-hospital mortality, it has become clear that longer-term mortality outcomes after these interventions in patients with multivessel disease are critically important. Major regional and institutional database analyses have documented substantial survival improvements after multivessel intervention in CABG cohorts versus PCI and have documented that these improvements in survival increase over time after intervention.16–19 Techniques that allow coupling of clinical data sets with Medicare administrative data to assess longitudinal outcomes are being developed; incorporation of these long-term mortality outcomes into the risk-prediction models will soon follow. Third, techniques for linking hospital and outpatient financial data with clinical data have been developed. Once incorporation of long-term outcomes data is accomplished, the prediction of clinical outcomes in the context of financial effectiveness (or ineffectiveness) going forward will be possible. This will be extremely important as new, multidisciplinary technologies are introduced into the clinical domain.
Indeed, this third implication has immediate relevance with respect to the new Medical Severity Diagnosis-Related Group (MS-DRG) system that Medicare has implemented recently.20 The identification and documentation of secondary diagnoses in patients with ischemic heart disease is an important component of this new MS-DRG system. A number of these secondary diagnoses and conditions have implications for preoperative risk prediction in these clinical models for PCI and CABG, as well as for postprocedural complications. Going forward, prediction of both clinical outcome risk and financial outcome risk across similar therapies will be critically important for patients, their providers, and the hospital systems in which these patients receive care.
Thus, in my judgment, the present report by Singh et al2 has both scientific and health policy relevance. Scientifically, this study is an important first step in broadening the use of risk models in cardiovascular disease. It is clearly in a patient’s best interest that providers convey this type of comprehensive, scientifically valid information about the relative risks of intervention before the choice of the intervention is made.
Limitations exist, however, and these are important. As the authors point out, differences in definitions and variable content can have an effect on the performance of such models when applied across different therapeutic database platforms. To this end, both the American College of Cardiology’s National Cardiovascular Data Registry and STS NCD leadership have worked extremely hard over the past year to resynchronize data definitions in both of these national databases. Performance of risk models at the extreme ends of predicted risk remains a challenge, although this model was relatively robust across most low- and high-risk subgroups. Now that this first step has been taken, however, these and other issues will be easier to address.
From a health policy perspective, the most important message in the present report is that the equitable evaluation of risk across different therapies can be incorporated at the bedside to improve the quality of care. Soon, this will include appropriateness and financial effectiveness metrics. This everyday use of robust clinical data, and the derived clinical outcomes, is perhaps the strongest argument today for not abandoning clinical data systems in favor of administrative data. This is certainly true in the cardiovascular disease domain.
 |
Acknowledgments |
|---|
Disclosures
None.
|
Footnotes |
|---|
The opinions expressed in this article are not necessarily those of the editors or of the American Heart Association.
|
References |
|---|
 Top References |
|---|
- The Pocket Oxford American Dictionary of Current English. New York, NY: Oxford University Press; 2002.
- Singh M, Gersh BJ, Li S, Rumsfeld JS, Spertus JA, O’Brien SM, Suri RM, Peterson ED. Mayo Clinic Risk Score for percutaneous coronary intervention predicts in-hospital mortality in patients undergoing coronary artery bypass graft surgery. Circulation. 2008; 117: 356–362.
[Abstract/Free Full Text] - O’Connor GT, Plume SK, Olmstead EM, Coffin LH, Morton JR, Maloney CT, Nowicki ER, Levy DG, Tryzelaar JF, Hernandez F, Adrian L, Casey KJ, Bundy D, Soule DN, Marrin CAS, Nugent WC, Charlesworth DC, Clough R, Katz S, Leavitt BJ, Wennberg JE; Northern New England Cardiovascular Disease Study Group. Multivariate prediction of in-hospital mortality associated with coronary artery bypass graft surgery. Circulation. 1992; 85: 2110–2118.
[Abstract/Free Full Text] - Hannan EL, Wu C, Bennett EV, Carlson RE, Culliford AT, Gold JP, Higgins RS, Isom OW, Smith CR, Jones RH. Risk stratification of in-hospital mortality for coronary artery bypass graft surgery. J Am Coll Cardiol. 2006; 47: 661–668.
[Abstract/Free Full Text] - Shroyer AL, Coombs LP, Peterson ED, Eiken MC, DeLong ER, Chen A, Ferguson TB Jr, Grover FL, Edwards FH. The Society of Thoracic Surgeons: 30-day operative mortality and morbidity risk models. Ann Thoracic Surg. 2003; 75: 1856–1864.
[Abstract/Free Full Text] - Nashef SA, Roques F, Michel P, Gauducheau E, Lemeshow S, Salamon R. European system for cardiac operative risk evaluation (EuroSCORE). Eur J Cardiothorac Surg. 1999; 16: 9–13.
[Abstract/Free Full Text] - Nashef SA, Roques F, Hammill BG, Peterson ED, Michel P, Grover FL, Wyse RK, Ferguson TB. Validation of European System for Cardiac Operative Risk Evaluation in North American cardiac surgery. Eur J Cardiothorac Surg. 2002; 22: 101–105.
[Abstract/Free Full Text] - Ferguson TB Jr, Hamill BG, Peterson ED, DeLong ER, Grover FL. A decade of change: risk profiles and outcomes for isolated coronary artery bypass grafting procedures, 1990–1999: a report from the STS National Database Committee and the Duke Clinical Research Institute. Ann Thorac Surg. 2002; 73: 480–489.
[Abstract/Free Full Text] - Society of Thoracic Surgeons National Database Spring 2007 executive summary. STS Web site. Available at: http:/www.sts.org/sections/stsnationaldatabase/publications/executive/article.html. Accessed November 12, 2007.
- EuroSCORE Web-based interactive risk calculator. EuroSCORE Web site. Available at: http://www.euroscore.org/calc.html. Accessed November 12, 2007.
- STS Web-based risk calculator. STS Web site. Available at: http://www.sts.org/sections/stsnationaldatabase/riskcalculator. Accessed November 12, 2007.
- Wu C, Hannan EL, Walford G, Ambrose JA, Holmes DR, King SB, Clark LT, Katz S, Sharma S, Jones RH. A risk score to predict in-hospital mortality for percutaneous coronary interventions. J Am Coll Cardiol. 2006; 47: 654–660.
[Abstract/Free Full Text] - Shaw RE, Anderson HV, Brindis RG, Krone RJ, Klein LW, McKay CR, Block PC, Shaw WJ, Hewitt K, Weintraub WS. Development of a risk adjustment mortality model using the American College of Cardiology– National Cardiovascular Data Registry (ACC-NCDR) experience: 1998–2000. J Am Coll Cardiol. 2002; 39: 1104–1112.
[Abstract/Free Full Text] - Parsonnet V, Dean D, Bernstein AD. A method of uniform stratification of risk for evaluating the results of surgery in acquired heart disease [published correction appears in Circulation. 1990;82:1078]. Circulation. 1989; 79 (pt 2): I-3–I-12.[Medline]
[Order article via Infotrieve]
- Mehta RH, Grab JD, O’Brien SM, Bridges CR, Gammie MS, Haan CK, Ferguson TB Jr, Peterson ED. Bedside tool for predicting the risk of postoperative dialysis in patients undergoing cardiac surgery. Circulation. 2006; 114: 2208–2216.
[Abstract/Free Full Text] - Smith PK, Califf RM, Tuttle RH, Shaw LK, Lee KL, DeLong ER, Lilly RE, Sketch MH, Peterson ED, Jones RH. Selection of surgical or percutaneous coronary intervention provides differential longevity benefit. Ann Thorac Surg. 2006; 82: 1420–1429.
[Abstract/Free Full Text] - Brenner SJ, Lytle BW, Casserly IP, Schneider JP, Topol EJ, Lauer MS. Propensity analysis of long-term survival after surgical or percutaneous revascularization in patients with multivessel coronary artery disease and high-risk features. Circulation. 2004; 109: 2290–2295.
[Abstract/Free Full Text] - Hannan EL, Racz MJ, Walford G, Jones RH, Ryan TJ, Bennett E, Culliford AT, Isom OW, Gold JP, Rose EA. Long-term outcomes of coronary artery bypass grafting versus stent implantation. N Engl J Med. 2005; 352: 2174–2183.
[Abstract/Free Full Text] - Malenka DJ, Leavitt BJ, Hearne MJ, Robb JF, Baribeau YR, Ryan TJ, Helm RE, Kellett MA, Dauerman HL, Dacey LJ, Silver MT, VerLee PN, Weldner PW, Hettleman BD, Olmstead EM, Piper WD, O’Connor GT. Comparing long-term survival of patients with multivessel coronary artery disease after CABG or PCI: analysis of BARI-like patients in Northern New England. Circulation. 2005; 112: 371–376.
- Medicare MS-DRG document. Fed Regist. 2007;72:24679–25135. Available at: http://www.cms.hhs.gov/quarterlyproviderupdates/downloads/cms1533p.pdf. Accessed November 12, 2007.
| ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||