doi: 10.1161/CIRCULATIONAHA.105.561126
(Circulation. 2005;112:1088-1091.)
© 2005 American Heart Association, Inc.
Editorial |
Measuring Percutaneous Coronary Intervention Quality by Simple Case Volume
From the Divisions of Cardiology and Clinical Biometrics, Brigham and Women’s Hospital and Harvard Medical School (R.E.K.), and the Department of Health Care Policy, Harvard Medical School, and the Department of Biostatistics, Harvard School of Public Health (S.T.N.), Boston, Mass.
Correspondence to Dr Richard E. Kuntz, Division of Clinical Biometrics, Brigham and Women’s Hospital, 75 Francis St, Boston, MA 02155. E-mail rkuntz{at}partners.org
Key Words: Editorials • stents • angioplasty • catheterization • coronary disease
 |
Introduction |
|---|
 Top Introduction Volume-Outcome Measures and...Complexity of Volume Outcome...PCI and Outcomes of...References |
|---|
Monitoring health care and developing metrics to gauge optimum systems from suboptimum ones is a multidimensional effort that spans the spectrum of health services research and administration. The quality assessment tools for operative procedures range from policies requiring measurement of general compliance within medical society guidelines, in terms of medical indications and procedural techniques, to local hospital peer review and medical executive committee oversight focused on case review. Although these quality measures are certainly labor intensive and require the highest level of objective reasoning and judgment, simpler metrics have been proposed that offer better ease of use and objectivity. Probably the most practical of the simpler methods for operative procedures is the measurement of periodic operative count, usually by physician and hospital, and a comparison of this count to a specified number or threshold felt to signify the highest quality. Inherent in this comparison of volume and quality is the operation-specific assessment of volume and outcomes to support the notion of a positive or negative (ie, not flat) relationship. In virtually all analyses in which a significant relationship has been demonstrated, an inverse relationship between volume and adverse outcomes was seen.1,2 Such relationships have also been seen for percutaneous coronary interventions (PCI), and the American College of Cardiology and American Heart Association (AHA/ACC) Joint Task Force guidelines for PCI have recommended minimum threshold goals for hospitals and operators to ensure the highest quality of PCI care.3
See p 1171
Hannan et al have analyzed the New York State Percutaneous Coronary Interventions Reporting System (PICRS) database for all 34 nonfederal hospitals and all 264 PCI operators from 1998 to 2000.4 The authors draw on this comprehensive and unique reporting system and its high ascertainment to provide adjusted analysis for comparison of expected and observed adverse events. They have concluded that the ACC/AHA recommended guidelines of a minimum of 75 cases per year per operator and 400 cases per year per hospital distinguish good from poor outcome rates in their volume-outcome analysis of in-hospital mortality and cardiac surgery. Combined with yet-to-be-determined quality process measures, these volume thresholds may help steer case referrals for PCI to higher volume centers until such process measures, such as analyses of techniques and adjunctive medications, can be used to improve outcomes for the lower-volume operators and hospitals. The present study is the first to evaluate stents in this regard, and is consistent with prior nonstent PCI therapy.
The present study by Hannan adds to the growing evidence of higher volume and lower adverse event associations for operations in general, and for PCI in particular. The volume-outcome analysis methods and end point choices for these analyses are complex, however, and strict interpretation of the exact count cutoffs employed in these analyses as the threshold for health policies should be made with caution. Rather, the comparative methods and end points used for this and other volume-outcome studies certainly support the notion that higher volume for operators and hospitals is desirable, but limitations in these methods should soften the studied cutoffs into relative zones for threshold policies.
|
Volume-Outcome Measures and Quality Assessment |
|---|
|
TopIntroductionVolume-Outcome Measures and...Complexity of Volume Outcome...PCI and Outcomes of...References |
|---|
Evidence supporting volume of procedures as a valuable metric for assessing quality of care is based on the reduction in variance of observed adverse outcomes by volume as an explanatory variable, first shown for several operations by Luft in 1979,5 and more recently for coronary bypass surgery.1,6,7 Overviews of the literature on volume-outcome relationships for cardiac and other procedures, considered to be at high risk, have shown consistent relationships between higher volume and lower adverse events.1,2,8 A recent study of surgery in 2.5 million Medicare patients also showed an inverse relationship between volume and adverse outcomes, in which the lowest volume-adverse events ratios were seen in some cancer operations such as pancreatectomy and esophagectomy, whereas ratios for cardiac or carotid surgery were closer to unity.9 In contrast to proposed PCI thresholds, cutoff volume counts for esophagectomy and pancreatectomy that demonstrated differential outcomes were 1 to 5 procedures per year versus >11 procedures per year.10
The evidence for significant volume-outcome relationships for cardiac procedures was first rooted in cardiac surgery, in which administrative databases (with limitations in adjustment methods) showed modest improvements in mortality reduction with increasing hospital volume.9 Hannan was the first to use more sophisticated risk adjustment techniques in a specialized database (New York Cardiac Surgery Reporting System), and showed a more striking differential in volume-outcomes: 7.25% mortality for hospitals performing <200 operations per year as compared with 2.85% mortality rate for hospitals performing >890 operations per year.11 Extension of this methodology to PCI in the balloon angioplasty era also demonstrated improved clinical outcomes with higher volume.12,13 Thus, when assessing the quality of PCI based on the dimension of procedure volume-mortality, the findings of the present study by Hannan are consistent with the inverse volume-adverse event relationship already seen in other operative procedures, including cardiac surgery and nonstent PCI.
|
Complexity of Volume Outcome Analyses in the Assessment of Healthcare Quality |
|---|
|
TopIntroductionVolume-Outcome Measures and...Complexity of Volume Outcome...PCI and Outcomes of...References |
|---|
Given the wide range of healthcare delivery patterns that have direct implications on surgical volume by operator or hospital, dictated by local demographics, geography, payor mix and variable practice patterns, the assumption that high volume is always better for patients would have tremendous impact on attaining strategies for optimum health care. Hannan et al4 state that other process measures for quality should be explored and validated to understand the low volume–high adverse event rate relationship, and that one should act on these measures to improve outcomes for low-volume operators and hospitals. Alternatively, the inverse volume-outcome relationship may lead to the adoption of regionalized healthcare programs that concentrate surgical procedures at high-volume centers for high-volume operators. Such regionalization may be disruptive for many patients and healthcare providers. Therefore, with each additional study such as Hannan et al4 supporting the theme that higher PCI volume is associated with improved clinical outcomes, with its inevitable policy consequences, critical reviews of these reports are required.
The calculation of volume-outcome ratios is straightforward, but the inference made about the quality of surgical operations and procedures based on volume of cases is complex. The intuitive notion that practice makes perfect, often cited as a mechanism of observed inverse volume-outcome associations,14 may drive the expectation of an inverse volume-adverse outcome relationship for each new study. There are many confounders that may act to cause such a result, obscuring the real cause of higher adverse outcome rates. For example, some operators and hospitals may be more apt to take on complicated cases that are associated with higher inherent risk of the adverse event measured. Such a relationship may also be associated with volume, and thus confound the observed volume-outcome result. Hannan et al4 have used a uniquely comprehensive database and proper adjustment techniques to minimize this potential bias. Moreover, the use of 3 outcomes, in-hospital death, same day, and same stay coronary bypass surgery, help to corroborate the conclusion. Of course, the outcomes measured should have direct relationships with our understanding of quality of PCI. Furthermore, the measured thresholds in the present analysis by Hannan (75 cases for operators and 400 cases for hospitals) should be interpreted cautiously because an optimization analysis for the best thresholds was not formally conducted, and the variance inherent in low-volume sample sizes make analysis of thresholds difficult.
Shahian and Normand pointed out that in the analysis of volume-outcomes for operative procedures, the low-volume estimates of adverse event rates are associated with high variance because of shrinking sample sizes.1 Several figures from Hannon et al show smoothed regression curves with inflection points that are taken as meaningful thresholds for determining cutoffs for high-quality care. The X-on-Y local regression technique employed, however, is fraught with sufficient wobble because of high variability associated with low sample size that makes direct interpretations of these inflection points notoriously difficult. An example of this result is shown in the Figure, in which simulated data via a fixed complication rate of 0.91% across a wide range of case volume resulted in substantial wobble of the local smoother regressor, giving an apparent inflection point between 200 and 400 patients. Therefore, threshold inference made on such analyses should be interpreted with caution because of the noise associated with low-volume samples. Alternatively, the prespecified 2-group analyses based on the recommendations of the ACC/AHA task force guidelines are valid, but the exact number needed to provide competence is not likely summarized by a single cutoff or the ones used in this analysis. A principled approach to identification and estimation of a volume cutoff involves not only the estimation of the cutoff but also the uncertainty attached to such cutoff. Thus, the volume-outcome relationship should be viewed as a general concept, with cutoffs for quality that only approximate those analyzed.
|
|
PCI and Outcomes of Interest |
|---|
|
TopIntroductionVolume-Outcome Measures and...Complexity of Volume Outcome...PCI and Outcomes of...References |
|---|
PCI is likely one of the most evolving and changing fields in medicine. During its
28-year history, elective PCI has never been shown to extend life or prevent death—its value has been found in the improvement in quality of life through reduction of myocardial ischemia. Several hundred randomized controlled trials have been performed in the PCI arena. As evidenced in recent drug-eluting stent trials,15 the primary end points and principal secondary end points of elective coronary device trials are either composite end points that are mainly driven by clinical restenosis components or explicit measure of restenosis, such as angiographic renarrowing or target lesion revascularization. Virtually all new PCI devices have been valued on the basis of successful durability of the index revascularization, generally measured as freedom from restenosis. Likewise, trials of adjunctive medical therapy for elective PCI measure acute death and coronary bypass rates, but the principal end points are mainly driven by the rates of acute myocardial infarction. Such outcome analysis have caused the sea change in stent therapy, which has adopted the more expensive drug-eluting stent rather than the less expensive bare metal stent as the treatment of choice in >80% of PCI cases in just 2 years. Measurements of acute death and emergent bypass surgery are important end points that are carefully measured in contemporary elective PCI device trials but have been relegated as safety measures assumed to be equivalent between the standard and new promising treatment. In other words, the improvements in PCI stent technology during the past 10 years and our assumptions of consequential patient benefit have rarely been driven by reductions in acute death or emergent or urgent bypass surgery. Whether measurement of PCI operative competency and quality should be studied the same way as is contemporary coronary device and medication trials is questionable, however. In the present study by Hannan et al,4 the case mix includes some nonelective cases of acute myocardial infarction PCI, in which the interest in acute mortality is higher in contemporary PCI myocardial infarction trials. The vast majority of cases in the current study are elective, and the measurement of and quality inference made about acute mortality may be challenged as the optimum analysis. Of course, limitations in cost and resources make measurement of follow-up end points nearly impossible.
This question of whether the quality of PCI by operators or hospitals should be measured as in the clinical trials of new stent treatments has challenged us in employing a research program to evaluate PCI at hospitals without cardiac surgery on-site in Massachusetts. That the other end points used in PCI device and drug trials may be of interest, in addition to in-hospital mortality and coronary bypass surgery, has been examined carefully. For this specific question of quality (elective PCI at hospitals without surgical backup), we feel that PCI quality may not be faithfully measured by acute in-hospital mortality and coronary bypass alone, as good- and poor-quality PCI can be performed without these events. An alternative measure of quality may be the level of safe and durable complete revascularization, manifested by freedom from downstream interventions. We have thus promoted a primary end point similar to that used in the coronary stent trials of freedom from major adverse cardiac events, which includes the need for late-term (1 year) repeat revascularization of both the index lesions and other lesions. Thus, a comparison of patients randomized to sites with and without surgical backup will be compared on such a composite end point, mainly driven by out-of-hospital late-term follow-up events. Similarly, one could advance the argument that other PCI quality endeavors may also be measured with a combination of similar acute and late-term outcomes.
The findings of Hannan et al4 that the higher volume of cases for both operators and hospitals is associated with lower adverse events (in-hospital mortality and CABG) come from a well-designed study that adds to the general notion that operative procedure healthcare quality may be summarized by case volume. The thresholds used in this analysis provide rough guidelines for policy decisions about PCI, and future PCI quality analysis may be focused on determining optimum thresholds and their corresponding estimates of uncertainty rather than on proving the inverse volume-outcome relationship already shown by Hannan and colleagues.
|
Acknowledgments |
|---|
Disclosure
Drs Kuntz and Normand are members of the Massachusetts Cardiac Care Quality Commission and Dr Normand leads a data-coordinating center that examines the quality of PCI care.
|
Footnotes |
|---|
The opinions expressed in this article are not necessarily those of the editors or of the American Heart Association.
|
References |
|---|
|
TopIntroductionVolume-Outcome Measures and...Complexity of Volume Outcome...PCI and Outcomes of...References |
|---|
1. Shahian DM, Normand ST. The volume-outcome relationship: from Luft to Leapfrog. Ann Thorac Surg. 2003; 75: 1048–1058.
2. Halm EA, Lee C, Chassin MR. How is volume related to quality in health care? A systematic review of the research literature. In: Hewitt M, ed. Interpreting the Volume-Outcome Relationship in the Context of Health Care Quality: Workshop Summary. Washington, DC: National Academies Press; 2000.
3. Smith SC, Dove JT, Jacobs AK, Kennedy JW, Keriakes D, Kern MJ, Kuntz RE, Popma JJ, Schaff HV, Williams DO. ACC/AHA Guidelines for Percutaneous Coronary Intervention. Circulation. 2001; 103: 3019–3041.
4. Hannan EL, Wu C, Walford G, King SB III, Holmes DR Jr, Ambrose JA, Sharma S, Katz S, Clark LT, Jones RH. Volume-outcome relationships for percutaneous coronary interventions in the stent era. Circulation. 2005; 112: 1171–1179.
5. Luft HS, Bunker JP, Enthoven AC. Should operations be regionalized? The empirical relationship between surgical volume and mortality. N Engl J Med. 1979; 301: 1364–1369.[Abstract]
6. Luft HS, Romano PS. Chance, continuity and change in hospital mortality rates. Coronary artery bypass graft patients in California hospitals. JAMA. 1993; 270: 331–337.
7. Hannan EL, O’Donnell JF, Kilburn H Jr, Bernard HR, Yazici A. Investigation of the relationship between volume and mortality for surgical procedures performed in New York State hospitals. JAMA. 1989; 262: 503–510.
8. Dudley RA, Johansen KL, Brand R, Rennie DJ, Milstein A. Selective referral to high-volume hospitals: estimating potentially avoidable deaths. JAMA. 2000; 283: 1159–1166.
9. Birkmeyer JD, Siewers AE, Finlayson EV, Stukel TA, Lucas FL, Batista I, Welch HG, Wennberg DE. Hospital volume and surgical mortality in the United States. N Engl J Med. 2002; 346: 1128–1137.
10. Begg CB, Cramer LD, Hoskins WJ, Brennan MF. Impact of hospital volume on operative mortality for major cancer surgery. JAMA. 1998; 280: 1747–1751.
11. Hannan EL, Kilburn H Jr, Brnard H, O’Donnell JF, Lukacik G, Shields EP. Coronary artery bypass surgery: the relationship between inhospital mortality rate and surgical volume after controlling for clinical risk factors. Med Care. 1991; 29: 1094–1107.[CrossRef][Medline] [Order article via Infotrieve]
12. Jollis JG, Peterson ED, DeLong ER, Mark DB, Collins SR, Muhlbaier LH, Pryor DB. The relation between the volume of coronary angioplasty procedures at hospitals treating Medicare beneficiaries and short-term mortality. N Engl J Med. 1994; 331: 1625–1629.
13. Hannan EL, Racz M, Ryan TJ, McCallister BD, Johnson LW, Arani DT, Guerci AD, Sosa J, Topol EJ. Coronary angioplasty volume-outcome relationships for hospitals and cardiologists. JAMA. 1997; 277: 892–898.
14. Luft HS, Hunt SS, Maerki SC. The volume-outcome relationship: practice-makes-perfect or selective-referral patterns? Health Serv Res. 1987; 22: 157–182.[Medline] [Order article via Infotrieve]
15. Moses JW, Leon MB, Popma JJ, Fitzgerald PJ, Holmes DR, O’Shaughnessy C, Caputo RP, Kereiakes DJ, Williams DO, Teirstein PS, Jaeger JL, Kuntz RE. Sirolimus-eluting stents versus standard stents in patients with stenosis in a native coronary artery. N Engl J Med. 2003; 349: 1315–1323.
Related Article:
Circulation 2005 112: 1171-1179.
This article has been cited by other articles:
 |
|
 |
|
  L. W. Klein, P. Kolm, X. Xu, R. J. Krone, H. V. Anderson, J. S. Rumsfeld, R. G. Brindis, and W. S. Weintraub A Longitudinal Assessment of Coronary Interventional Program Quality: A Report From the American College of Cardiology-National Cardiovascular Data Registry J. Am. Coll. Cardiol. Intv., February 1, 2009; 2(2): 136 - 143. [Abstract] [Full Text] [PDF]
|
|
|
|
 |
|
 J. A. Bittl Reply J. Am. Coll. Cardiol., July 18, 2006; 48(2): 416 - 416. [Full Text] [PDF]
|
|
| |||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||