This Article Abstract Alert me when this article is cited Alert me if a correction is posted Citation Map Services Email this article to a friend Similar articles in this journal Similar articles in PubMed Alert me to new issues of the journal Download to citation manager Request Permissions Citing Articles Citing Articles via HighWire Citing Articles via Google Scholar Google Scholar Articles by Aaronson, K. D. Articles by Mancini, D. M. Search for Related Content PubMed PubMed Citation Articles by Aaronson, K. D. Articles by Mancini, D. M.

(Circulation. 1997;95:2660-2667.)
© 1997 American Heart Association, Inc.

Articles

Development and Prospective Validation of a Clinical Index to Predict Survival in Ambulatory Patients Referred for Cardiac Transplant Evaluation

Keith D. Aaronson, MD, MS; J. Sanford Schwartz, MD; Tze-Ming Chen, BS; Kar-Lai Wong, MD; James E. Goin, PhD; Donna M. Mancini, MD

From the Division of Circulatory Physiology, Heart Failure and Cardiac Transplant Programs, College of Physicians and Surgeons, Columbia University (K.D.A., D.M.M.), New York, NY, and the Division of General Medicine and the Leonard Davis Institute for Health Economics (J.S.S.), University of Pennsylvania School of Medicine (J.S.S., T.-M.C., K.-L.W., J.E.G.), Philadelphia.

Correspondence to Dr Keith D. Aaronson, Division of Cardiology, University of Michigan Medical Center, Taubman 3910, Ann Arbor, MI 48109-0366. E-mail keith{at}umich.edu

	Abstract

Top Abstract Introduction Methods Results Discussion References

 
Background Risk stratification of patients with end-stage congestive heart failure is a critical component of the transplant candidate selection process. Accurate identification of individuals most likely to survive without a transplant would facilitate more efficient use of scarce donor organs.

Methods and Results Multivariable proportional hazards survival models were developed with the use of data on 80 clinical characteristics from 268 ambulatory patients with advanced heart failure (derivation sample). Invasive and noninvasive models (with and without catheterization-derived data) were constructed. A prognostic score was determined for each patient from each model. Stratum-specific likelihood ratios were used to develop three prognostic-score risk groups. The models were prospectively validated on 199 similar patients (validation sample) by calculation of the area under the receiver operating characteristic curve for 1-year event-free survival, the censored c-index for event-free survival, and comparison of event-free survival curves for prognostic-score risk strata. Outcome events were defined as urgent transplant or death without transplant. The noninvasive model performed well in both samples, and increased performance was not attained by the addition of catheterization-derived variables. Prognostic-score risk groups derived from the noninvasive model in the derivation sample effectively stratified the risk of an outcome event in both samples (1-year event-free survival for derivation and validation samples, respectively: low risk, 93% and 88%; medium risk, 72% and 60%; high risk, 43% and 35%).

Conclusions Selection of candidates for cardiac transplantation may be improved by use of this noninvasive risk-stratification model.

Key Words: heart failure • survival • risk factors • predictive models

	Introduction

Top Abstract Introduction Methods Results Discussion References

 
Cardiac transplantation is an effective treatment option for patients with severe CHF. Though originally considered only for patients with NYHA class IV symptoms, the 82% and 74% 1- and 3-year heart transplantation survival rates compare favorably to the 15% to 20% annual mortality rates for patients with class III heart failure (References 1, 1^{1 1} a, and UNOS Scientific Registry data as of October 7, 1995, UNOS transplantation information web site). As a result, an increasing number of ambulatory patients with advanced CHF are placed on transplant waiting lists while the supply of donor organs remains limited and fixed. Therefore, accurate identification of patients most likely to benefit from transplantation is imperative.²

We previously showed that ambulatory heart failure patients who can achieve a peak O₂ >14 mL·kg^-1·min^-1 are at low risk for cardiac mortality and can have cardiac transplantation safely deferred. In contrast, 52% of patients with a peak O₂ 14 mL·kg^-1·min^-1 died or underwent urgent transplantation within 1 year, and these patients are now often placed on transplantation waiting lists.³

However, risk stratification based solely on peak O₂ is limited. Such an approach does not make efficient use of routinely obtained clinical measures of known prognostic significance.^{4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19} We hypothesized that pretransplant risk stratification in ambulatory patients with advanced heart failure could be improved with the use of a predictive model incorporating multiple independent predictors of mortality. Two models were developed. One model used all data routinely collected, including invasively obtained hemodynamic measurements (the invasive model). The second model was based solely on noninvasively obtained clinical measures (the noninvasive model), which logistically and financially would be more widely applicable. Both models were then prospectively validated in a temporally and geographically distinct set of patients.

	Methods

Top Abstract Introduction Methods Results Discussion References

 
The model derivation sample contained data collected from 268 ambulatory patients aged <70 years with LVEF 40% referred to HUP for evaluation of severe heart failure and/or cardiac transplant evaluation between July 1986 and December 1991. The model validation sample consisted of data from 199 ambulatory patients aged 70 years with LVEF 40% who were referred to CPMC for cardiac transplant evaluation between July 1993 and July 1995. All patients who were able to perform an exercise test unlimited by angina or claudication were enrolled. All patients gave informed consent. The study was approved by institutional review boards at HUP and CPMC.

Age, sex, NYHA class, resting heart rate, serum sodium, and cause of heart failure were similar for the two groups. Patients in the derivation sample were more frequently white, had lower LVEF and peak O₂, had higher mean blood pressure, and were less likely to have an IVCD than patients in the validation sample. Loop diuretics, digoxin, and ACE inhibitors were used in a large majority of patients (Table 1). The median daily dose of captopril was 75 mg.

View this table: [in this window] [in a new window]   Table 1. Characteristics of the Model Derivation (n=268) and Model Validation (n=199) Samples

Clinical history and physical examination, blood chemistries, ECG, chest roentgenogram, radionuclide ventriculogram, exercise testing, and, when clinically indicated, right heart catheterization and coronary angiography were prospectively obtained on all patients after stabilization with maximal medical therapy (diuretics titrated to resolution of edema short of significant prerenal uremia [blood urea nitrogen>50 mg/dL]; ACE inhibitors titrated to target dose of captopril 50 mg TID or its equivalent as renal function and symptoms permitted; and digoxin in the absence of AV nodal dysfunction) (Table 2). A modified Charlson comorbidity index was computed for each patient by excluding the CHF and myocardial infarction categories so that only noncardiac diseases remained.²⁰

View this table: [in this window] [in a new window]   Table 2. Clinical Characteristics Evaluated for Potential Inclusion in the Predictive Model

Mean resting blood pressure was estimated as diastolic pressure plus pulse pressure as measured by auscultation. Maximal treadmill exercise testing with measurement of peak O₂ was performed during treadmill exercise using a modified Naughton protocol and a metabolic cart (Sensor-Medics).³ Percent of predicted maximal O₂ was determined as previously described.²¹ Right heart catheterization was performed with thermodilution catheters, with measurement of right atrial and pulmonary artery pressures, PCWP, and cardiac output. LVEF was measured by radionuclide or contrast ventriculography.

Patients were followed up prospectively. Outcome events were defined as death without transplant or UNOS 1 transplant (ie, receiving mechanical or inotropic support before transplantation). For patients who remained alive and nontransplanted, follow-up was discontinued on January 1, 1993, at HUP and on October 1, 1995, at CPMC. Follow-up was complete in 97% and 99% of patients at HUP and CPMC, respectively.

Model Development
A series of univariable analyses were performed in the derivation sample to identify potentially important predictors of survival for evaluation in subsequent multivariable analyses. Initial univariable descriptive analyses were performed by use of the Kaplan-Meier method and log-rank tests.^{22 23} Patients who underwent UNOS 2 transplant were censored at transplant, as were patients alive without a transplant at the end of follow-up. Significant univariable predictors (P<.15) and other variables found to be significant in previous studies were analyzed with univariable and multivariable Cox proportional hazards models.²⁴ The proportional hazards assumption was confirmed graphically.²⁵

Two multivariable modeling strategies were used to develop candidate models: a stepwise forward-entry (P.05)/backward-elimination (P.05) selection process and a best-subset selection process that determined models with the highest ² statistic score for up to 11 explanatory variables (to maintain 10 outcome events per explanatory variable [109 outcome events occurred in the derivation sample]).²⁶ Candidate models were also formed by applying both of the aforementioned selection methods after specifying variables to be included that are believed to represent different aspects of the pathophysiology of heart failure.²⁷ The goal was to select the smallest number of explanatory variables needed to accurately predict survival in the derivation sample.^{27 28 29} To explore relationships between the variables selected for the models, Spearman's correlation coefficients were calculated. A prognostic score, the HFSS, was calculated for each patient as the absolute value of the sum of the products of the identified prognostic variables and their computed coefficients (ie, |ß₁x₁+ß₂x₂+...+ß_nx_n|, where x₁, x₂,...x_n are the values for the explanatory variables and ß₁, ß₂,...ß_n are the coefficients [ie, weights] assigned to each variable).²⁴

The ability of each candidate model to discriminate between patients who did and did not experience a study outcome was assessed in two ways: by calculation of the AUC for development of an outcome event at 1 year (excluding patients with censored follow-up at <1 year) and by calculation of the censored c-index for development of an outcome event at any time during follow-up.^{30 31} AUCs for different models were compared by the method of Hanley and McNeil.³² The c-index is an estimate of the probability that of two randomly selected patients, the patient with the higher HFSS will live free of an outcome event for longer than the patient with the lower HFSS.³¹ The censored c-index differs from the 1-year AUC in that it continues to differentiate between outcome events occurring after 1 year of follow-up and is able to consider censored events that occur at <1 year of follow-up.

SSLRs, the relative odds of an outcome event at 1 year for each stratum of HFSS compared with that of the entire cohort, were used to determine HFSS threshold values at which the probability of 1-year survival substantially increased or decreased. HFSS strata were initially formed at 0.1-point increments, and SSLRs and 95% CIs were calculated.³³ By combining adjacent strata with statistically indistinct SSLRs, threshold values for HFSS were determined.³³ Kaplan-Meier curves were then estimated and plotted for each HFSS risk stratum in the derivation sample.

Model Validation
The final model and HFSS risk strata from the derivation sample were then prospectively validated. The HFSS was calculated for each patient in the validation sample from the final model. Model discrimination in the validation sample was determined by calculating the AUC for 1-year survival and the c-index. Discrimination of the HFSS risk strata was tested in the validation sample by computing SSLRs and Kaplan-Meier curves for the HFSS strata in the validation sample. All statistical testing was two-tailed. Calculations were performed with SAS version 6.09 and Microsoft Excel version 4.0.

	Results

Top Abstract Introduction Methods Results Discussion References

 
Patient Outcome
After the initial evaluation, 81 of 268 patients at HUP (derivation sample) and 101 of 199 patients at CPMC (validation sample) were listed for transplantation. Compared with patients rejected for transplant, listed patients had lower peak O₂, LVEF, mean BP, and serum sodium and higher NYHA class and resting heart rate (P<.05 for each sample). Survival curves for the derivation and validation groups are shown in Fig 1. Freedom from an outcome event was significantly better for the derivation group than for the validation group (76±3% versus 68±4% at 1 year and 63±3% versus 51±5% at 2 years, respectively; P<.025).

View larger version (13K): [in this window] [in a new window]   Figure 1. Freedom from an outcome event for the derivation (HUP) and validation (CPMC) samples. The number of patients remaining available for follow-up for each sample is shown below the graph.

Noninvasive and Invasive Predictive Models
Preliminary analyses identified many variables at least marginally significant as predictors of an outcome event (P.15) in univariable analyses (displayed in italics in Table 2). A noninvasive predictive model was selected containing the following seven variables: ischemic cardiomyopathy, resting heart rate, LVEF, IVCD (QRS duration 0.12 second of any cause), mean resting blood pressure, peak O₂, and serum sodium (Table 3). The invasive predictive model, based on the 231 patients in the derivation sample for whom right heart catheterization was performed, also included mean PCWP (Table 3). Although other clinical characteristics significantly contributed to other candidate noninvasive and invasive models (eg, S₃, log of the duration of heart failure, presence of a pacemaker, and cardiac index), the addition of these variables to the selected model did not enhance discrimination in the derivation sample.

View this table: [in this window] [in a new window]   Table 3. Noninvasive Model (n=268) and Invasive Model (n=231) Obtained From the Derivation Sample

Spearman's correlation coefficients between the five continuous variables of the noninvasive model were relatively weak, ranging from 0.12 to 0.22 (sign ignored). Correlations were statistically significant for only 5 of 10 pairs.

Discrimination for the noninvasive and invasive models was similar, and therefore additional analyses are shown for the noninvasive model only (Table 4). Univariable models using the eight variables from which the noninvasive and invasive models were composed exhibited significantly worse discrimination by AUC in the derivation sample than did the multivariable models (all P<.002). In the validation sample, discrimination of each of the univariable models was inferior to the multivariable models (all P<.036), with the exception of the peak O₂ model, which performed similar to the noninvasive model (P=.88 for comparison of AUCs). However, peak O₂ was among the worst univariable predictors in the derivation sample (AUC=0.62±0.04; c-index=0.61±0.05). The inconsistent discrimination of peak O₂ in the two samples was shared by most univariable predictors and contrasted with the relatively consistent performance of the noninvasive model.

View this table: [in this window] [in a new window]   Table 4. Discrimination of Noninvasive, Invasive, and Univariable Models in the Derivation and Validation Samples

HFSS Risk Strata and Survival
SSLRs and 95% CIs for 1-year event-free survival for ranges of HFSS (Table 5) revealed three distinct strata: low risk (HFSS 8.10), medium risk (HFSS 7.20 to 8.09), and high risk (HFSS 7.19). The odds of an outcome event at 1 year for the low-risk stratum were 5 and 21 times less than for the medium- and high-risk strata. Event-free survival rates at 1 year for the low-, medium-, and high-risk HFSS strata were 93±2%, 72±5%, and 43±7%, respectively (Fig 2, left). The HFSS strata for the noninvasive model provided highly effective risk stratification throughout the entire follow-up period (P<.0001 overall and for each pairwise comparison between groups). Event-free survival rates for the medium- and high-risk strata were much worse than would be expected after cardiac transplantation; the low-risk stratum had an event-free survival rate that was better than would be expected with transplantation.

View this table: [in this window] [in a new window]   Table 5. SSLRs With 95% CI for Freedom From an Outcome Event at 1 Year for HFSS Strata, Developed From the Derivation Sample From the Noninvasive Model, as Assessed in Both the Derivation and Validation Samples

View larger version (12K): [in this window] [in a new window]   Figure 2. Event-free survival for the low-, medium-, and high-prognostic-score risk groups for the derivation sample (top left) and for the validation sample (top right). The number of patients remaining available for follow-up for each risk group is shown below each graph.

SSLRs for 1-year event-free survival for the validation sample were similar to those obtained from the derivation sample (Table 5). Forty-four percent of patients in each sample were in the low-risk stratum. The event-free survival rate at 1 year in the validation sample was 88±4%, 60±6%, and 35±10% in the low-, medium-, and high-risk strata, respectively (Fig 2, right). Throughout the entire follow-up period, event-free survival was significantly better for the low- versus the medium-risk group (P<.0001) and for the low- versus the high-risk group (P<.0001).

	Discussion

Top Abstract Introduction Methods Results Discussion References

 
The aim of the present study was to develop a more accurate prognostic tool for ambulatory patients with advanced heart failure. Clinical measures routinely obtained in the evaluation of patients with advanced heart failure were combined into clinical indices that effectively stratify the mortality risk for heart failure patients. This is the first study in which a clinical decision-making tool for the prediction of survival has been prospectively applied and validated in a separate group of patients with advanced heart failure.

We evaluated 80 clinical characteristics in a large sample of patients with advanced heart failure for potential inclusion in our multivariable predictive models. Not surprisingly, about half of these clinical characteristics were significant univariable predictors of survival. Prior studies^{3 5 6 7 8 12 14 15 16 34 35 36 37 38} have demonstrated the prognostic value of each of the variables included in the noninvasive model. However, risk assessment based on any single factor has limited accuracy and reproducibility. Individual predictors often conflict and are only weakly correlated.¹⁵ Only by combining individual clinical characteristics into a multivariable predictive index can the frequently discordant implications of multiple univariable analyses be made coherent.

Retrospective, multivariable analyses of data from large heart failure samples has led to the identification of a number of independent predictors of survival.^{4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19} Limitations of earlier studies include inappropriate target populations, performance before therapy with ACE inhibitors was prevalent, evaluation of a limited set of clinical characteristics, the nonuse of formal statistical methods of identifying thresholds, and lack of allowance for calculation of an individual patient's risk. No model has been prospectively validated in an independent sample of patients.

We developed noninvasive and invasive models that differed only in the addition of PCWP to the invasive model. Because there was no statistical advantage afforded by the use of the invasive model, we recommend that the noninvasive model be used to assess heart failure mortality risk. By avoiding the time, expense, and risk of right heart catheterization, use of the noninvasive model should allow more efficient and cost-effective pretransplant risk stratification. Right heart catheterization to assess the risk of right-sided circulatory failure after transplant could be reserved for those patients found to have heart failure severe enough to warrant a transplant on the basis of the noninvasive model.³⁹

As expected, both multivariable models exhibited significantly better discrimination than did any single-variable model except for the peak O₂ model, which performed similarly in the validation sample only. However, peak O₂ was one of the weaker univariable predictors when evaluated in the derivation sample. In fact, although discrimination by AUC for the noninvasive model was relatively consistent in the two samples, discrimination of the univariable models sometimes varied quite widely between samples. Resting heart rate, the best predictor in the derivation sample, performed particularly poorly in the validation sample. By incorporating multiple weakly correlated variables, the noninvasive model is likely to be more robust than any single clinical characteristic when prospectively applied.

Statistical programs for proportional hazards modeling provide a variety of automated variable selection strategies. We used clinical judgment to guide variable selection and included variables incorporating multiple aspects of heart failure pathophysiology: myocardial ischemia (ischemic cardiomyopathy), systolic dysfunction (LVEF), diastolic dysfunction (PCWP), activation of the renin-angiotensin-aldosterone system (serum sodium), activation of the sympathetic nervous system (resting heart rate), myocardial injury/fibrosis (IVCD), and more integrative measures (peak O₂ and mean blood pressure). It is likely that the performance of the final models we selected was enhanced by this approach.²⁹

The most useful clinical tests have one or more distinct threshold values at which the likelihood of the outcome of interest markedly changes. By evaluating the likelihood ratios associated with small ranges of the HFSS, three risk strata were identified. The odds of experiencing an outcome event during the first year of follow-up for patients in the high-risk stratum was 12 to 21 times that of patients in the low-risk stratum; for patients in the medium-risk stratum, the odds of an event were 5 times as great as for patients in the low-risk stratum.

Limitations
Predictive models often perform less well when applied to a new set of patients. The statistical techniques that underlie these models attempt to make sense of the clinical information in the derivation sample, whether that information is truly clinically meaningful or simply "noise" (eg, random variability or measurement error). Some degree of deterioration with prospective validation is expected. Despite this, when we tested the models prospectively, we found only a mild loss of discrimination for the noninvasive model.

Wasson et al⁴⁰ proposed methodological standards for creating and validating clinical prediction rules that should enhance future performance when met by the original investigators and heeded by subsequent users. These standards were largely met in our study. Predictive findings and outcome events were not assessed independently in this study because the same investigators performed both tasks. However, the use of objective variables as predictors and death as an outcome minimized the risk of ascertainment bias.

For this investigation, we enrolled ambulatory patients with advanced heart failure who presented to specialized clinics at either of two urban tertiary care centers who were able to perform a stress test. These models are likely to perform well in comparable patients treated with a loop diuretic, digoxin, and an ACE inhibitor. In the future, ß-adrenergic blockers may assume a prominent role in the treatment of heart failure.^{41 42 43 44} By lowering resting heart rate and raising blood pressure and LVEF, therapy with ß-blockers would be expected to improve prognostic scores. Only 10% and 11% of patients in the derivation and validation samples, respectively, received a ß-blocker. We suspect that our models will retain important prognostic information for patients treated with ß-blockers, but this will need to be tested.

Patients in the medium- and high-risk strata at HUP and CPMC would be expected to have improved survival with cardiac transplantation. Survival for medium-risk patients could be better than was observed in our study if the overall survival rate were better than in our samples. A bayesian analysis suggests that if the overall 1-year survival rate for a sample is 83%, a medium-risk group with an SSLR of 0.80 (0.84 and 0.75 in our samples) would have a 1-year event-free survival rate of 80%, a rate low enough that transplant listing should be considered.

On the basis of our earlier work, investigators at both clinical centers used peak O₂ measurements to guide the selection of potential transplant candidates. This raises the important issue of whether this component of the model did not predict outcome but rather determined it. Although a low peak O₂ would have increased the likelihood of placement on the waiting list and might therefore have increased the likelihood of nonurgent transplant, it would not have increased the likelihood of a study outcome (death or UNOS 1 transplant). Furthermore, by censoring at the time of UNOS 2 transplantation, the study design biases against peak O₂ as an important predictor of survival.

Multivariable models containing some univariable predictors not included in the final models performed nearly as well as these models when cross-validated in the derivation sample. Because of differences between study populations and chance, some clinical characteristics that we excluded might be included if this process was replicated by others. Type II errors are likely in some cases (eg, only six patients in the derivation sample had 50% stenosis of the left main coronary artery).

Our analysis was limited to data routinely collected in the course of patient evaluations during the study period. Several potentially important predictors of survival were not assessed. Data from Holter monitor recordings and signal-averaged ECGs and measurements of serum neurohormones and cytokines might have improved the predictive models.^{5 6 10 38 45 46 47 48 49 50} We did not obtain hemodynamic measurements after attempting to optimize acute hemodynamics with diuretics and vasodilators, as suggested by Stevenson.^{19 51} Whether such measurements would provide independent risk stratification beyond that provided by our present models will require further investigation.

Clinical Implications
A multivariable model incorporating clinical data routinely collected noninvasively in the evaluation of patients with advanced heart failure can stratify their risk of adverse outcome. By using the HFSS and associated risk strata of the noninvasive model, clinicians caring for these patients can more effectively select candidates for cardiac transplantation. Patients in medium- and high-risk groups are most likely to die or require urgent transplant in the following year; they should be considered for cardiac transplantation if no contraindications are present. Transplantation can be safely deferred in patients in the low-risk group. This approach should facilitate more efficient use of scarce donor hearts and selection of high-risk patients for enrollment in clinical trials of new heart failure therapies.

	Selected Abbreviations and Acronyms

 

AUC = area under the receiver operating characteristic curve CHF = congestive heart failure CPMC = Columbia-Presbyterian Medical Center HFSS = heart failure survival score HUP = Hospital of the University of Pennsylvania IVCD = intraventricular conduction delay LVEF = left ventricular ejection fraction NYHA = New York Heart Association PCWP = pulmonary capillary wedge pressure peak O2 = peak exercise oxygen consumption SSLR = stratum-specific likelihood ratio UNOS = United Network for Organ Sharing

	Acknowledgments

 
This work was supported by grants from the NIH, National Center for Research Resources (RR-00040 and RR-00645), and by a Clinical Investigator Development Award (5-K08-HL02829) from the NHLBI (Dr Aaronson). The authors wish to thank Frank Harrell, PhD, for providing the FORTRAN code for calculating the censored c-index and John Pierce, MD, MS, for providing an Excel macro for calculating SSLRs. Physicians interested in using the HFSS may obtain a copy of an Excel macro that performs the calculations by sending a formatted 3.5-in diskette (note Macintosh or PC) and a self-addressed envelope with appropriate postage to Dr Aaronson.

Received September 16, 1996; revision received March 26, 1997; accepted April 2, 1997.

	References

Top Abstract Introduction Methods Results Discussion References

 

1. Konstam M, Dracup K, Brooks N, Dacey R, Dunbar S, Jackson A, Jessup M, Johnson J, Jones R, Luchi R, Massie B, Pitt B, Rose E, Rubin L, Wright R. Heart Failure: Evaluation and Care of Patients with Left Ventricular Systolic Dysfunction—Clinical Practice Guideline No. 11. Rockville, Md: Agency for Health Care Policy and Research; June 1994. US Dept of Health and Human Services publication No. 94-0612.
   
2. 1995 Annual Report of the US Scientific Registry for Transplant Recipients and the Organ Procurement and Transportation Network. Data 1988-1994. UNOS, Richmond, Va, and the Division of Transplantation, Bureau of Health Resources Development, Health Resources, and Services Administration. US Department of Health and Human Services, Rockville, Md.
   
2. Stevenson L, Warner S, Steimle A, Fonarow G, Hamilton M, Moriguchi J, Kobashigawa J, Tillisch J, Drinkwater D. The impending crisis awaiting cardiac transplantation: modeling a solution based on selection. Circulation. 1994;89:450-457.[Abstract/Free Full Text]
   
3. Mancini D, Eisen H, Kussmaul W, Mull R, Edmunds L, Wilson J. Value of peak exercise oxygen consumption for optimal timing of cardiac transplantation in ambulatory patients with heart failure. Circulation. 1991;83:778-786.[Abstract/Free Full Text]
   
4. Wilson J, Schwartz J, Sutton M, Ferraro N, Horowitz L, Reichek N, Josephson M. Prognosis in severe heart failure: relation to hemodynamic measurements and ventricular ectopic activity. J Am Coll Cardiol. 1983;2:403-410.[Medline] [Order article via Infotrieve]
   
5. Cohn J, Levine T, Olivari M, Garberg V, Lura D, Francis G, Simon A, Rector T. Plasma norepinephrine as a guide to prognosis in patients with chronic congestive heart failure. N Engl J Med. 1984;311:819-823.[Abstract]
   
6. Unverferth D, Magorien R, Moeschberger M, Baker P, Fetters J, Leier C. Factors influencing the one-year mortality of dilated cardiomyopathy. Am J Cardiol. 1984;54:147-152.[Medline] [Order article via Infotrieve]
   
7. Likoff MJ, Chandler S, Kay H. Clinical determinants of mortality in chronic congestive heart failure secondary to idiopathic dilated or to ischemic cardiomyopathy. Am J Cardiol. 1987;59:634-638.[Medline] [Order article via Infotrieve]
   
8. Cohn J, Rector T. Prognosis of congestive heart failure and predictors of mortality. Am J Cardiol. 1988;62:25A-30A.[Medline] [Order article via Infotrieve]
   
9. Gradman A, Deedwania P, Cody R, Massie B, Packer M, Pitt B, Goldstein S, for the Captopril-Digoxin Study Group. Predictors of total mortality and sudden death in mild to moderate heart failure. J Am Coll Cardiol. 1989;14:564-570.[Abstract]
   
10. Rockman H, Juneau C, Chatterjee K, Rouleau J-L. Long-term predictors of sudden and low output death in chronic congestive heart failure secondary to coronary artery disease. Am J Cardiol. 1989;64:1344-1348.[Medline] [Order article via Infotrieve]
    
11. Keogh A, Baron D, Hickie J. Prognostic guides in patients with idiopathic or ischemic dilated cardiomyopathy assessed for cardiac transplantation. Am J Cardiol. 1990;65:903-908.[Medline] [Order article via Infotrieve]
    
12. Kelly T, Cremo R, Nielson C, Shabetai R. Prediction of outcome in late-stage cardiomyopathy. Am Heart J. 1990;119:1111-1121.[Medline] [Order article via Infotrieve]
    
13. Griffin B, Shah P, Ferguson J, Rubin S. Incremental prognostic value of exercise hemodynamic variables in chronic congestive heart failure secondary to coronary artery disease or to dilated cardiomyopathy. Am J Cardiol. 1991;67:848-853.[Medline] [Order article via Infotrieve]
    
14. Parameshwar J, Keegan J, Sparrow J, Sutton G, Poole-Wilson P. Predictors of prognosis in severe chronic heart failure. Am Heart J. 1992;123:421-426.[Medline] [Order article via Infotrieve]
    
15. Cohn J, Johnson G, Shabetai R, Loeb H, Tristani F, Rector T, Smith R, Fletcher R, for the V-HeFT VA Cooperative Studies Group. Ejection fraction, peak exercise oxygen consumption, cardiothoracic ratio, ventricular arrhythmias, and plasma norepinephrine as determinants of prognosis in heart failure. Circulation. 1993;87(suppl VI):VI-5-VI-16.
    
16. Campana C, Gavazzi A, Berzuini C, Larizza C, Marioni R, D'Armini A, Pederzolli N, Martinelli L, Vigano M. Predictors of prognosis in patients awaiting heart transplantation. J Heart Lung Transplant. 1993;12:756-765.[Medline] [Order article via Infotrieve]
    
17. Anguita M, Arizon J, Bueno G, Latre J, Sancho M, Torres F, Gimenez D, Concha M, Valles F. Clinical and hemodynamic predictors of survival in patients aged <65 years with severe congestive heart failure secondary to ischemic or nonischemic dilated cardiomyopathy. Am J Cardiol. 1993;72:413-417.[Medline] [Order article via Infotrieve]
    
18. Madsen B, Keller N, Christiansen E, Christensen N. Prognostic value of plasma catecholamines, plasma renin activity, and plasma atrial natriuretic peptide at rest and during exercise in congestive heart failure: comparison with clinical evaluation, ejection fraction, and exercise capacity. J Cardiac Failure. 1995;1:207-216.[Medline] [Order article via Infotrieve]
    
19. Stevenson L, Couper G, Natterson B, Fonarow G, Hamilton M, Woo M, Creaser J. Target heart failure populations for new therapies. Circulation. 1995;92(suppl II):II-174-II-181.
    
20. Charlson M, Pompei P, Ales K, MacKenzie C. A new method of classifying prognostic comorbidity in longitudinal studies: development and validation. J Chronic Dis. 1987;40:373-383.[Medline] [Order article via Infotrieve]
    
21. Aaronson K, Mancini D. Is percent of predicted maximal O₂ a better predictor of survival than peak O₂ for patients with severe heart failure? J Heart Lung Transplant. 1995;14:981-989. Published erratum appears in J Heart Lung Transplant. 1996;15:106-107.[Medline] [Order article via Infotrieve]
    
22. Kaplan E, Meier P. Nonparametric estimation from incomplete observations. J Am Stat Assoc. 1958;53:457-481.
    
23. Peto R, Peto L. Asymptotically efficient rank invariant procedures. J R Stat Soc. 1972;135:185-207.
    
24. Cox D. Regression models and life-tables (with discussion). J R Stat Soc. 1972;34:187-202.
    
25. Lee E. Statistical Methods for Survival Data Analysis. New York, NY: John Wiley & Sons Inc; 1992:250-263.
    
26. Harrell FE Jr, Lee KL, Matchar DB, Reichert TA. Regression models for prognostic prediction: advantages, problems, and suggested solutions. Cancer Treat Rep. 1985;69:1071-1077.[Medline] [Order article via Infotrieve]
    
27. Hosmer D, Lemeshow S. Applied Logistic Regression. New York, NY: John Wiley & Sons Inc; 1989:82-83.
    
28. Charlson M, Ales K, Simon R, MacKenzie R. Why predictive indexes perform less well in validation studies: is it magic or methods? Arch Intern Med. 1987;147:2155-2161.[Abstract]
    
29. Harrell F, Lee K, Califf R, Pryor D, Rosati R. Regression modelling strategies for improved prognostic prediction. Stat Med. 1984;3:143-152.[Medline] [Order article via Infotrieve]
    
30. Hanley J, McNeil B. The meaning and use of the area under a receiver operating characteristic (ROC) curve. Radiology. 1982;143:29-36.[Abstract/Free Full Text]
    
31. Harrell F, Califf R, Pryor D, Lee K, Rosati R. Evaluating the yield of diagnostic tests. JAMA. 1982;247:2543-2546.[Abstract]
    
32. Hanley J, McNeil B. A method of comparing the areas under receiver operating characteristic curves derived from the same cases. Radiology. 1983;148:839-843.[Abstract/Free Full Text]
    
33. Pierce J, Cornell R. Integrating stratum-specific likelihood ratios with the analysis of ROC curves. Med Decis Making. 1993;13:141-151.
    
34. Franciosa JA, Wilen M, Ziesche S, Cohn JN. Survival in men with severe chronic left ventricular failure due to either coronary heart disease or idiopathic dilated cardiomyopathy. Am J Cardiol. 1983;51:831-836.[Medline] [Order article via Infotrieve]
    
35. Szlachcic J, Massie B, Kramer B, Topic N, Tubau J. Correlates and prognostic implication of exercise capacity in chronic congestive heart failure. Am J Cardiol. 1985;55:1037-1042.[Medline] [Order article via Infotrieve]
    
36. Cohn J, Archibald D, Ziesche S, Franciosa J, Harston W, Tristani F, Dunkman S, Jacobs W, Francis G, Flohr K, Goldman S, Cobb F, Shah P, Saunders R, Fletcher R, Leob H, Hughes V, Baker B. Effect of vasodilator therapy on mortality in chronic congestive heart failure: results of a Veterans Administration Cooperative Study. N Engl J Med. 1986;314:1547-1552.[Abstract]
    
37. Lee W, Packer M. Prognostic importance of serum sodium concentration and its modification by converting-enzyme inhibition in patients with severe chronic heart failure. Circulation. 1986;73:257-267.[Abstract/Free Full Text]
    
38. Mancini D, Wong K, Simson M. Prognostic value of an abnormal signal-averaged electrocardiogram in patients with nonischemic congestive cardiomyopathy. Circulation. 1993;87:1083-1092.[Abstract/Free Full Text]
    
39. Costard-Jackle A, Fowler M. Influence of preoperative pulmonary artery pressure on mortality after heart transplantation: testing of potential reversibility of pulmonary hypertension with nitroprusside is useful in defining a high risk group. J Am Coll Cardiol. 1992;19:48-54.[Abstract]
    
40. Wasson J, Sox J, Neff R, Goldman L. Clinical prediction rules: applications and methodological standards. N Engl J Med. 1985;313:793-799.[Abstract]
    
41. Waagstein F, Bristow M, Swedberg K, Camerini F, Fowler M, Silver M, Gilbert E, Johnson M, Goss F, Hjalmarson A. Beneficial effects of metoprolol in idiopathic dilated cardiomyopathy: Metoprolol in Dilated Cardiomyopathy (MDC) Trial Study Group. Lancet. 1993;342:1441-1446.[Medline] [Order article via Infotrieve]
    
42. CIBIS Investigators and Committees. A randomized trial of ß-blockade in heart failure: the Cardiac Insufficiency Bisoprolol Study (CIBIS). Circulation. 1994;90:1765-1773.[Abstract/Free Full Text]
    
43. Sackner-Bernstein J, Mancini D. Rationale for treatment of patients with chronic heart failure with adrenergic blockade. JAMA. 1995;274:1462-1467.[Abstract]
    
44. Packer M, Bristow M, Cohn J, Colucci W, Fowler M, Gilbert E, Shusterman N. Effect of carvedilol on morbidity and mortality in chronic heart failure: US Carvedilol Heart Failure Study Group. N Engl J Med. 1996;334:1349-1355.[Abstract/Free Full Text]
    
45. Follansbee WP, Michelson EL, Morganroth J. Nonsustained ventricular tachycardia in ambulatory patients: characteristics and association with sudden cardiac death. Ann Intern Med. 1980;92:741-747.
    
46. Meinertz T, Hofmann T, Kasper W, Treese N, Bechtold H, Stienen U, Pop T, Leitner E, Andresen D, Meyer J. Significance of ventricular arrhythmias in idiopathic dilated cardiomyopathy. Am J Cardiol. 1984;53:902-907.[Medline] [Order article via Infotrieve]
    
47. Gottlieb S, Kukin M, Ahern D, Packer M. Prognostic importance of atrial natriuretic peptide in patients with chronic heart failure. J Am Coll Cardiol. 1989;13:1534-1539.[Abstract]
    
48. Levine B, Kalman J, Mayer L, Fillit HM, Packer M. Elevated circulating levels of tumor necrosis factor in severe chronic heart failure. N Engl J Med. 1990;323:236-241.[Abstract]
    
49. Swedberg K, Eneroth P, Kjekshus J, Wilhelmsen L, for the CONSENSUS Trial Study Group. Hormones regulating cardiovascular function in patients with chronic congestive heart failure and their relation to mortality. Circulation. 1990;82:1730-1736.[Abstract/Free Full Text]
    
50. Ferrari R, Bachetti T, Confortini R, Opasich C, Febo O, Corti A, Cassani G, Visioli P. Tumor necrosis factor soluble receptors in patients with various degrees of congestive heart failure. Circulation. 1995;92:1479-1486.[Abstract/Free Full Text]
    
51. Stevenson L. Tailored therapy before transplantation for effective treatment of advanced heart failure: effective use of vasodilators and diuretics. J Heart Lung Transplant. 1991;10:468-476.[Medline] [Order article via Infotrieve]
    

This article has been cited by other articles:

				B. Heidecker, E. K. Kasper, I. S. Wittstein, H. C. Champion, E. Breton, S. D. Russell, M. M. Kittleson, K. L. Baughman, and J. M. Hare Transcriptomic Biomarkers for Individual Risk Assessment in New-Onset Heart Failure Circulation, July 15, 2008; 118(3): 238 - 246. [Abstract] [Full Text] [PDF]

				L. A. Allen, J. E. Yager, M. J. Funk, W. C. Levy, J. A. Tulsky, M. T. Bowers, G. C. Dodson, C. M. O'Connor, and G. M. Felker Discordance Between Patient-Predicted and Model-Predicted Life Expectancy Among Ambulatory Patients With Heart Failure JAMA, June 4, 2008; 299(21): 2533 - 2542. [Abstract] [Full Text] [PDF]

				C. W. Yancy Predicting Life Expectancy in Heart Failure JAMA, June 4, 2008; 299(21): 2566 - 2567. [Full Text] [PDF]

				J. C. Matthews, T. M. Koelling, F. D. Pagani, and K. D. Aaronson The right ventricular failure risk score a pre-operative tool for assessing the risk of right ventricular failure in left ventricular assist device candidates. J. Am. Coll. Cardiol., June 3, 2008; 51(22): 2163 - 2172. [Abstract] [Full Text] [PDF]

				I. S. Anand, T. S. Rector, M. Kuskowski, S. Thomas, N.J. Holwerda, and J. N. Cohn Effect of Baseline and Changes in Systolic Blood Pressure Over Time on the Effectiveness of Valsartan in the Valsartan Heart Failure Trial Circ Heart Fail, May 1, 2008; 1(1): 34 - 42. [Abstract] [Full Text] [PDF]

				R. S. Gardner, K. S. Chong, E. O'Meara, A. Jardine, I. Ford, and T. A. McDonagh Renal dysfunction, as measured by the modification of diet in renal disease equations, and outcome in patients with advanced heart failure Eur. Heart J., December 2, 2007; 28(24): 3027 - 3033. [Abstract] [Full Text] [PDF]

				M. Demir, M. Kanadasi, O. Akpinar, Y. Donmez, M. Avkarogullari, C. Alhan, T. Inal, M. San, A. Usal, and M. Demirtas Cardiac Troponin T as a Prognostic Marker in Patients With Heart Failure : A 3-Year Outcome Study Angiology, November 1, 2007; 58(5): 603 - 609. [Abstract] [PDF]

				T. J. Wang Significance of Circulating Troponins in Heart Failure: If These Walls Could Talk Circulation, September 11, 2007; 116(11): 1217 - 1220. [Full Text] [PDF]

				T. Breidthardt, M. Christ, M. Matti, D. Schrafl, K. Laule, M. Noveanu, T. Boldanova, T. Klima, W. Hochholzer, A. P Perruchoud, et al. QRS and QTc interval prolongation in the prediction of long-term mortality of patients with acute destabilised heart failure Heart, September 1, 2007; 93(9): 1093 - 1097. [Abstract] [Full Text] [PDF]

				R. Arena, J. Myers, M. A. Williams, M. Gulati, P. Kligfield, G. J. Balady, E. Collins, and G. Fletcher Assessment of Functional Capacity in Clinical and Research Settings: A Scientific Statement From the American Heart Association Committee on Exercise, Rehabilitation, and Prevention of the Council on Clinical Cardiology and the Council on Cardiovascular Nursing Circulation, July 17, 2007; 116(3): 329 - 343. [Full Text] [PDF]

				A. Casaleggio, R. Maestri, M. T. L. Rovere, P. Rossi, and G. D. Pinna Prediction of sudden death in heart failure patients: a novel perspective from the assessment of the peak ectopy rate Europace, June 1, 2007; 9(6): 385 - 390. [Abstract] [Full Text] [PDF]

				M. Kosiborod, G. E. Soto, P. G. Jones, H. M. Krumholz, W. S. Weintraub, P. Deedwania, and J. A. Spertus Identifying Heart Failure Patients at High Risk for Near-Term Cardiovascular Events With Serial Health Status Assessments Circulation, April 17, 2007; 115(15): 1975 - 1981. [Abstract] [Full Text] [PDF]

				J. Butler and A. Leon The Elusive Scourge of Sudden Cardiac Death: Is Rational Decision Making Possible? Should There Be Standards of Risks and Predictions in Medicine? J. Am. Coll. Cardiol., April 3, 2007; 49(13): 1434 - 1435. [Full Text] [PDF]

				I. Goldenberg, A. J. Moss, S. McNitt, W. Zareba, W. J. Hall, M. L. Andrews, and for the MADIT-II Investigators Inverse Relationship of Blood Pressure Levels to Sudden Cardiac Mortality and Benefit of the Implantable Cardioverter-Defibrillator in Patients With Ischemic Left Ventricular Dysfunction J. Am. Coll. Cardiol., April 3, 2007; 49(13): 1427 - 1433. [Abstract] [Full Text] [PDF]

				C. M. Boyd, C. O. Weiss, J. Halter, K. C. Han, W. B. Ershler, and L. P. Fried Framework for Evaluating Disease Severity Measures in Older Adults With Comorbidity J. Gerontol. A Biol. Sci. Med. Sci., March 1, 2007; 62(3): 286 - 295. [Abstract] [Full Text] [PDF]

C. E. Canter, R. E. Shaddy, D. Bernstein, D. T. Hsu, M. R.K. Chrisant, J. K. Kirklin, K. R. Kanter, R. S.D. Higgins, E. D. Blume, D. N. Rosenthal, et al. Indications for Heart Transplantation in Pediatric Heart Disease: A Scienti