doi: 10.1161/CIRCULATIONAHA.107.701722
| ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
(Circulation. 2007;115:2599-2601.)
© 2007 American Heart Association, Inc.
Editorial |
Reevaluation of the Volume-Outcome Relationship for Pediatric Cardiac Surgery
From the Department of Cardiology, Children’s Hospital; Department of Pediatrics, Harvard Medical School; and Department of Biostatistics, Harvard School of Public Health, Boston, Mass.
Correspondence to Kimberlee Gauvreau, ScD, Department of Cardiology, Children’s Hospital, 300 Longwood Ave, Boston, MA 02115. E-mail gauvreau{at}tch.harvard.edu
Key Words: Editorials • heart surgery, congenital • hospitals
Over the last 3 decades, a growing body of work has demonstrated a relationship between volume and outcome for a variety of complex surgical procedures and medical conditions. Two large-scale reviews have systemically evaluated the methodology and results of many of these studies across a broad range of conditions.1,2 In 2000, Dudley et al1 found that, among 128 studies examining 40 different procedures or conditions, 80% reported a statistically significant relationship between higher institutional case volume and better clinical outcomes; none of the published studies reported an association between higher volume and worse outcomes. In 2002, Halm et al2 evaluated 135 studies for 27 procedures or conditions, many of which were included in the previous review; a preliminary version of this report served as the focus of an Institute of Medicine workshop examining the volume-outcome relationship in the context of healthcare quality.3 The authors concluded that 70% of comparisons examining either institutional or physician case volume noted a statistically significant relationship between higher volume and better outcomes. Once again, none of the reports found the opposite to be true.
Article p 2652
Pediatric cardiac surgery is the only surgical procedure performed in children for which a volume-outcome relationship has been documented. In 4 large studies conducted in the United States (2 studies4,5 included in the review by Dudley et al,1 3 studies4–6 in the Halm et al2 review, and 1 study7 published subsequent to these reviews), larger annual surgical case volumes were associated with substantial reductions in in-hospital mortality. These results were quite consistent despite the fact that the studies used different data sources, different patient selection, different risk adjustment methods, and different definitions of volume. Jenkins et al4 used aggregated hospital discharge data from California (1988) and Massachusetts (1989) to demonstrate a significantly higher risk-adjusted odds of in-hospital mortality for centers performing 
300 cases per year relative to those performing >300 cases (odds ratio [OR], 7.7, <10 cases; OR, 2.9, 10 to 100 cases; OR, 3.0, 101 to 300 cases). Using prospectively collected clinical data from New York State (1992–1995), Hannan et al5 found a significantly higher risk-adjusted in-hospital mortality rate at hospitals performing <100 cases per year compared with those performing 
100 cases (8.26% versus 5.95%). Like Hannan and colleagues, Sollano et al6 examined data from New York State (1990–1995) and found a statistically significant effect of volume (OR, 0.944 for every 100 additional cases); it was noted that this relationship was most pronounced for neonates (age <31 days; OR, 0.636 for every 100 additional cases) and patients 31 days to 1 year of age (OR, 0.720). Chang and Klitzner7 used hospital discharge data from California (1995–1997) and identified mean annual volume cut points of 70 and 170 cases per year. Surgical cases were stratified as low or high risk on the basis of principal procedure code. For low-risk cases, the odds of in-hospital mortality were higher for medium-volume institutions (70 to 169 cases) than for high-volume institutions (170 cases; OR, 1.54); the odds also were higher for small-volume centers (10 to 69 cases; OR, 1.24), but this difference did not achieve statistical significance. For high-risk cases, the odds were higher for both medium-volume (OR, 2.54) and small-volume (OR, 2.67) centers relative to large-volume institutions.
In this issue of Circulation, the study by Bazzani and Marcin8 reevaluates the volume-outcome relationship for pediatric cardiac surgery using a larger, more contemporary hospital discharge database (1998–2003) from the state of California. This database contains patient discharge information from all hospitals in the state, excluding certain federal and Shriners hospitals. The authors began by carefully replicating the work of each of the 4 previously published reports; they then analyzed the relationship between volume and mortality using a newly developed risk adjustment model incorporating elements from each of the previous studies.
Replicating the study by Jenkins et al,4 Bazzani and Marcin again found higher risk-adjusted odds of in-hospital death for centers performing 300 cases per year relative to those performing >300 cases (OR, 3.04, <10 cases; OR, 1.61, 10 to 100 cases; OR, 1.45, 101 to 300 cases), although the ORs were lower than in the original study and the difference between the very smallest and largest centers did not achieve statistical significance. Reproducing the work by Hannan et al,5 they found that risk-adjusted mortality rates were similar for hospitals performing <100 cases per year relative to those performing 100 cases (3.87% versus 3.56%). Replicating the study of Sollano et al,6 they found that volume was no longer significantly associated with mortality in the population overall (OR, 0.99 for every 100 additional cases); however, there was a modest but statistically significant effect among neonates (OR, 0.97 for every 100 additional cases). Reproducing the work by Chang and Klitzner,7 they showed that the odds of in-hospital mortality were higher for low-volume institutions relative to high-volume centers for both low-risk (OR, 1.52) and high-risk (OR, 1.84) cases; medium-volume centers did not differ significantly from large ones.
After replicating the 4 original studies, Bazzani and Marcin incorporated features of each of the 4 models into a new predictive model for in-hospital mortality. All patient-level variables identified as potentially important by any of the previous authors were considered for inclusion. Annual surgical volume was treated as a continuous variable, and individual institutions were allowed to vary in volume from year to year. Annual volume also was dichotomized at 75 cases per year because California Children’s Services’ guidelines recommend that centers perform a minimum of 75 pediatric congenital open heart surgeries per year. In this new model, the authors found that increasing case volume again was associated with lower mortality (OR, 0.86 for every 100 additional cases; 95% CI, 0.81 to 0.92). In addition, hospitals performing 75 cases per year had a lower odds of mortality than hospitals performing <75 cases annually (OR, 0.75; 95% CI, 0.55 to 1.02), although this difference did not quite achieve statistical significance at the 0.05 level. Discrimination of the model was adequate, and calibration was good. When examining model diagnostics, however, the authors observed that data points with the highest annual case volumes also had the highest leverage, meaning that they had the strongest influence on the model results. When the single largest hospital was removed—the only institution in the data set with an annual surgical volume >400 cases—there was a decrease in both magnitude and statistical significance of the volume-outcome relationship (OR, 0.93 for every 100 additional cases; 95% CI, 0.82 to 1.05). The OR for centers performing 75 versus <75 cases per year also decreased (OR, 0.84; 95% CI, 0.62 to 1.16).
Bazzani and Marcin conclude that, although a volume-outcome relationship may still exist for pediatric cardiac surgery, it is neither as strong nor as consistent as had been observed in previous work. Other studies investigating selected adult cardiac surgeries have found that differences in outcomes between high- and low-volume centers have diminished over time.10–12 The authors speculate that a possible explanation for this result is that low-volume hospitals already avoid high-risk surgical procedures that they are not equipped to perform. Other explanations might include general advances in treatments and technology that were adopted at higher-volume centers sooner than lower-volume centers12 and the positive effects of quality improvement initiatives.10
Although it may indeed be true that the overall volume-outcome relationship in pediatric cardiac surgery is changing with time, other possibilities must be considered. First, we must consider the effect of decreasing in-hospital mortality on the magnitude of a potential volume-outcome relationship. As the overall risk of an outcome decreases, the OR comparing risk of the outcome between 2 groups will tend to decrease even if the incremental difference in risk between the groups remains the same. (This point can be illustrated by a simple example. If the mortality rate is 10% among high-volume hospitals and is 90% higher, or 19%, among low-volume hospitals, then the unadjusted OR for mortality is 2.11 for low versus high volume. Alternatively, if the mortality rate is 5% among high-volume hospitals and is again 90% higher, or 9.5%, among low-volume hospitals, the OR would be 1.99 for low versus high volume.) In the present study, the in-hospital mortality rate for the entire data set is 3.6%, which is considerably lower than the mortality rates in the 4 original studies; as a result, we would expect some reduction in the magnitude of the observed volume-outcome relationship simply because survival has improved. An overall decrease in the proportion of patients who die also would reduce the power of a study to identify statistically significant results.
Second, although the authors present valid reasons for treating volume as a continuous variable and for evaluating only 1 potential cut point, the dichotomization at 75 cases per year recommended by the California Children’s Services’ guidelines, it is still possible that the relationship between volume and mortality, or rather between volume and the logarithm of the odds of mortality, is not linear as assumed by their model. For example, outcomes might be different for institutions in the "tails" of the volume distribution, those centers with only the very lowest or the very highest case volumes. Although it is not explicitly discussed, the data presented in the present report suggest that the lowest-volume centers might still have substantially higher risk than high-volume centers. In the replication of the study by Jenkins et al,4 the odds of in-hospital mortality among institutions performing <10 cases was 3.04 times higher than the odds for centers performing >300 cases annually. In addition, in the reproduction of the work by Chang and Klitzner,7 the lowest-volume centers (10 to 69 cases per year) had significantly higher risk-adjusted mortality than the highest-volume centers for both low- and high-risk patients. Furthermore, there was only a single institution with an annual volume >400 cases in the data set analyzed; it is possible that the threshold at which risk-adjusted mortality is considerably lower lies near or above this cut point. This cannot be addressed by the use of data from California only; a more nationally representative database is needed. The fact that the replication of the Jenkins et al4 study, which set the cut point for high-volume centers at 300 cases per year, continues to find a statistically significant difference between high- and medium-volume institutions lends credence to this possibility.
Whether a volume-outcome relationship exists or not, most studies of pediatric cardiac surgery and other complex surgical procedures have noted that the trend for lower risk of death at larger institutions is not universal; wide variation has been observed.2,13–15 Some large institutions have higher-than-expected mortality rates, and many small institutions have lower-than-expected rates. This suggests that high surgical case volume may be a surrogate or proxy for aspects of care that are more likely to be provided at larger centers but are not necessarily exclusive to these centers. In this case, institution-specific risk-adjusted outcomes would be more informative than any simple volume threshold.
In summary, the study by Bazzani and Marcin raises the very interesting possibility that the volume-outcome relationship for pediatric cardiac surgical procedures has diminished over time. Additional investigation is needed to determine whether this finding holds up in other databases. The authors wisely encourage caution in interpreting volume-outcome relationships and recommend that future volume-outcome studies make use of statistical diagnostics to further explore any apparent association or lack of association.
 |
Acknowledgments |
|---|
Disclosures
None.
|
Footnotes |
|---|
The opinions expressed in this article are not necessarily those of the editors or of the American Heart Association.
|
References |
|---|
 Top References |
|---|
1. Dudley RA, Johansen KL, Brand R, Rennie DJ, Milstein A. Selective referral to high-volume hospitals: estimating potentially avoidable deaths. JAMA. 2000; 283: 1159–1166.
2. Halm EA, Lee C, Chassin MR. Is volume related to outcome in health care? A systemic review and methodologic critique of the literature. Ann Intern Med. 2002; 137: 511–520.
3. Hewitt M, for the Committee on Quality of Health Care in America and the National Cancer Policy Board. Interpreting the Volume-Outcome Relationship in the Context of Health Care Quality: Workshop Summary. Washington, DC: Institute of Medicine, National Academy of Sciences; 2000.
4. Jenkins KJ, Newburger JW, Lock JE, Davis RB, Coffman GA, Iezzoni LI. In-hospital mortality for surgical repair of congenital heart defects: preliminary observations of variation by hospital caseload. Pediatrics. 1995; 95: 323–330.
5. Hannan EL, Racz M, Kavey RE, Quaegbeur JM, Williams R. Pediatric cardiac surgery: the effect of hospital and surgeon volume on in-hospital mortality. Pediatrics. 1998; 101: 963–969.
6. Sollano JA, Gelijns AC, Moskowitz AJ, Heitjan DF, Cullinane S, Saha T, Chen JM, Roohan PJ, Reemtsma K, Shields EP. Volume-outcome relationships in cardiovascular operations: New York State, 1990–1995. J Thorac Cardiovasc Surg. 1999; 117: 419–428.
7. Chang RK, Klitzner TS. Can regionalization decrease the number of deaths for children who undergo cardiac surgery? A theoretical analysis. Pediatrics. 2002; 109: 173–181.
8. Bazzani LG, Marcin JP. Case volume and mortality in pediatric cardiac surgery patients in California, 1998–2003. Circulation. 2007; 115: 2652–2659.
10. Hannan EL, Siu AL, Kumar D, Kilburn H Jr, Chassin MR. The decline in coronary artery bypass graft surgery mortality in New York State: the role of surgeon volume. JAMA. 1995; 273: 209–213.
11. Sowden AJ, Deeks JJ, Sheldon TA. Volume and outcome in coronary artery bypass graft surgery: true association or artifact? BMJ. 1995; 311: 151–155.
12. Ho V. Evolution of the volume-outcome relation for hospitals performing coronary angioplasty. Circulation. 2000; 101: 1806–1811.
13. Welke KF, Barnett MJ, Vaughan Sarrazin MS, Rosenthal GE. Limitations of hospital volume as a measure of quality of care for coronary artery bypass graft surgery. Ann Thorac Surg. 2005; 80: 2114–2120.
14. Hannan EJ. The relationship between volume and outcome in health care. N Engl J Med. 1999; 340: 1677–1679.
15. Shahian DM, Normand S-LT. The volume-outcome relationship: from Luft to Leapfrog. Ann Thorac Surg. 2003; 75: 1048–1058.
Related Article:
Circulation 2007 115: 2591.
| ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||