(Circulation. 1997;95:405-410.)
© 1997 American Heart Association, Inc.
Articles |
Sex and Test Verification Bias
Impact on the Diagnostic Value of Exercise Echocardiography
the Division of Cardiovascular Diseases, Mayo Clinic, Rochester, Minn.
Correspondence to Veronique L. Roger, MD, Division of Cardiovascular Diseases, Mayo Clinic, 200 First St SW, Rochester, MN 55905. E-mail roger.veronique{at}mayo.edu
 |
Abstract |
|---|
 Top Abstract IntroductionMethodsResultsDiscussionReferences |
|---|
Background The use of exercise echocardiography for the diagnosis of coronary artery disease (CAD) has been validated in pilot studies but is not documented in clinical practice and in women comparatively with men. The objectives of this study were to determine the effects of sex and of test verification bias on the diagnostic performance of exercise echocardiography.
Methods and Results Three thousand six hundred seventy-nine consecutive patients (1714 women, 1965 men) who underwent an exercise echocardiographic study were studied; the observed sensitivity, specificity, and correct classification rate were calculated among 340 patients (244 men, 96 women) who underwent angiography; to study the effect of test verification bias, sensitivity and specificity were estimated for all patients who underwent exercise echocardiography including those not referred to angiography. In the angiographic group, the prevalence of CAD was 60% in women and 80% in men. The observed sensitivity and specificity of exercise echocardiography was 78% and 44% in men and 79% and 37% in women. After adjustment for test verification bias, the estimated sensitivity was lower in women (32% versus 42% in men), whereas specificity was similar in both sexes. The positive predictive value was lower in women (66%) compared with men (84%).
Conclusions In clinical practice, test verification bias results in a lower observed specificity and a higher sensitivity of exercise echocardiography. In women, positive predictive value and adjusted sensitivity are lower compared with that in men.
Key Words: echocardiography • exercise • sex • diagnosis
|
Introduction |
|---|
|
TopAbstractIntroductionMethodsResultsDiscussionReferences |
|---|
Exercise echocardiography is widely used to detect coronary artery disease (CAD); however, this test has been validated by pilot studies conducted in selected populations, predominantly men, with a high percentage of angiographic referrals.1 2 3 4 In practice, patients are less selected, and the decision to proceed with angiography depends on the results of the test itself as well as on patient characteristics. The impact of this clinically appropriate but statistically biased selection process is to markedly increase the sensitivity and reduce the specificity of any given test.5 6 7 This may distort the apparent value of the test particularly in subgroups in which referral bias may be more pronounced, such as women. This has not been analyzed for exercise echocardiography.
Noninvasive detection of CAD is particularly complex in women, who have a lower prevalence of CAD than men.8 According to the bayesian probability theory,9 noninvasive methods will show a decrease in positive predictive value when applied to lower prevalence groups; this has been demonstrated for treadmill exercise testing in women.10 11 12 In addition, intrinsic sex differences have been observed with exercise radionuclide angiography,13 14 a stress test modality that examines left ventricular mechanics, as exercise echocardiography does. It is therefore plausible that similar sex-specific responses would be observed with exercise echocardiography. Finally, several studies have shown sex differences in the referral patterns to coronary angiography with women being less likely to be referred to coronary angiography after a stress test.15 16 17 All of the above could theoretically contribute to a different diagnostic value of exercise echocardiography in women compared with men. It is important to study such differences to help clarify the role of stress echocardiography in women at a time when cost-effectiveness mandates justification of test utilization. Previous reports analyzed the accuracy of stress echocardiography only in women18 19 20 and thus could not examine whether it differed in men. This study was undertaken to test the following hypotheses: (1) the diagnostic value of the test differs in women compared with men, and (2) in clinical practice, the sensitivity and specificity of exercise echocardiography are different from those initially reported in validation studies because of test verification bias.
|
Methods |
|---|
|
TopAbstractIntroductionMethodsResultsDiscussionReferences |
|---|
Patient Population
When exercise echocardiography was implemented at our institution, all consecutive patients undergoing exercise echocardiography were prospectively enrolled in a database in which information on each patient's clinical presentation, risk factors, treadmill exercise test, and exercise echocardiographic results were collected. This allowed complete ascertainment of the proportion of patients tested who subsequently underwent coronary angiography and analysis of the relationship between such referral and exercise echocardiography results.
All patients who underwent exercise echocardiography between January 1991 and September 1994 were considered for inclusion in this analysis. The exclusion criteria were a prior history of myocardial infarction or revascularization procedure (coronary artery bypass grafting or angioplasty), left bundle branch block, or permanent pacemaker. The study group included 3679 patients. Of these, the angiographic group consisted of 340 patients who had coronary angiography within 1 year of exercise echocardiography without any intercurrent event. The remainder of the study population was labeled the nonangiographic group (n=3339).
Exercise Echocardiography
Patients underwent a symptom-limited treadmill exercise echocardiographic study with preexercise and postexercise imaging as previously described.21 22 The exercise ECG was classified as normal, ischemic, or nondiagnostic. Ischemia was defined by horizontal or downsloping ST-segment depression of 
0.1 mV occurring 80 ms after the J point as determined by conventional reading of the exercise ECG. Two-dimensional echocardiographic images were recorded at rest and immediately after exercise in the four standard views (parasternal long-axis and short-axis, apical four-chamber and two-chamber views). The studies were interpreted by experienced observers unaware of the results of the angiography, using a 16-segment model to analyze the regional wall motion abnormalities.23 The ejection fraction was determined according to previously validated and published techniques.24 25 The criteria used to identify CAD included wall motion abnormalities at rest or after exercise.1 2 3 4 A normal study was defined by the presence of normal wall motion at rest and a global increase in contractility with exercise. Results were classified as normal, equivocal, or abnormal; a positive test was defined as clearly abnormal test; a negative test included normal or equivocal results.
Coronary Angiography
Referral to angiography was carried out at the discretion of the patient's cardiologist. Each angiogram was interpreted by two experienced angiographers unaware of the results of the exercise echocardiogram. Conventional analysis with visual assessment was used. In addition, in a subset of 68 patients, electronic calipers were used, and the results of the two readings were compared. Readings were carried out at different times, and the angiographers were unaware of the results of the first interpretation or of the results of the exercise echocardiogram. Significant CAD was defined as narrowing of the luminal diameter of an epicardial vessel or major branch of 50%. Multivessel disease was defined by the presence of two or more stenoses of 50% of two or more vessels.
Statistical Analysis
Group characteristics were compared by t tests or 
2 tests. Sensitivity, specificity, and positive and negative predictive values were conventionally defined, comparing echocardiography with angiography. The correct classification rate was defined as the sum of the number of patients with true-positive and true-negative tests divided by the total number of patients. Verification bias occurs when the result of the test is used to decide which patients will receive the verification test; in this setting, positive tests are overrepresented in the patients who are verified, resulting in an increase in the sensitivity and a decrease in the specificity. Begg and Greenes26 described a method that corrects for selection bias by taking into account the verification process and estimates sensitivity and specificity in the entire population undergoing the test. By applying this method, we constructed logistic regression models to identify predictors of CAD, using the angiographic group. The variables considered for inclusion in the models were age, sex, history of typical angina, diabetes mellitus, smoking, hypertension, hypercholesterolemia and history of familial coronary disease, workload achieved, positive exercise ECG, and exercise echocardiography results. The predicted probability of CAD was calculated in the nonangiographic group with the use of the logistic regression model developed in the angiographic group, and these probabilities were summed separately over those with positive and those with negative exercise echocardiographic results. Estimates of sensitivity and specificity for the entire population were obtained by combining these sums with the observed frequencies in the angiographic groups. These calculations were performed separately in men and women, although the model was constructed in the pooled angiographic sample, with allowance for sex differences and interactions as appropriate. Several estimates of adjusted sensitivity and specificity were derived, through the use of various models, to examine robustness to model error. In addition, the final model was modified to vary the resulting prevalence in the nonangiographic group by varying only the intercept term in the model. To characterize the variability of the adjusted estimates, the boot strap method was used; this sampling process generates repeated samples with approximately the same sampling distribution as the original sample27 and allowed us to characterize the variability of the adjusted estimates of sensitivity and specificity using 95% CI.
The simplified method by Diamond,28 which assumes that the verification bias is solely the result of the preferential referral of positive tests to angiography, was also applied, and the results were examined in comparison to those obtained through the use of the Begg and Greenes method.
|
Results |
|---|
|
TopAbstractIntroductionMethodsResultsDiscussionReferences |
|---|
Angiographic Group
The median interval between the exercise echocardiographic study and the angiography was 6 days (range, 0 to 363 days). In 92% of the cases (312 of 340), angiography was performed within 6 months of exercise echocardiography. The baseline characteristics of the patients in the angiographic group are summarized in Tables 1 and 2
by sex, according to the presence or absence of CAD. Women were older than men but had similar frequencies of risk factors and typical angina pectoris (Table 3). In 68 patients (204 stenoses), both conventional reading (with visual estimates of percent stenosis) and electronic calipers were used independently. There was complete agreement between the two interpretations in 91% (185 of 204) of the stenoses analyzed. Therefore, conventional reading was used for the rest of the analysis.
|
|
|
CAD was present in 58 of the 96 women (22 single-vessel, 14 two-vessel, 22 three-vessel disease). Among the 244 men, 194 had CAD (57 single-vessel, 56 two-vessel, 81 three-vessel disease). The prevalence of disease was significantly higher in men (80%) than in women (60%) (P<.001).
Exercise Echocardiography Results
Eighty-seven percent of the patients exercised according to a Bruce protocol (a Naughton or modified Bruce protocol was used in the remaining patients). Images were of diagnostic quality in 95% of the patients. Postexercise images were acquired within 1.3±0.4 minutes after cessation of the exercise test. In the entire group of 340 patients, the sensitivity was 78% (95% CI, 73% to 83%), specificity was 41% (95% CI, 31% to 51%), positive predictive value was 79% (95% CI, 74% to 84%), and negative predictive value was 40% (95% CI, 30% to 50%) (Table 4).
|
Among the 312 patients who underwent angiography within 6 months of exercise echocardiography, the results were similar: the sensitivity was 80% (95% CI, 75% to 85%), specificity was 38% (95% CI, 27% to 49%), positive predictive value was 80% (95% CI, 75% to 85%), and negative predictive value was 38% (95% CI, 27% to 49%).
Results According to Sex
The sensitivity of stress echocardiography was similar in men and women, at 78% and 79%, respectively (P=.81). The specificity was 37% in women versus 44% in men (P=.5). The positive predictive value was 84% in men and 66% in women (P=.001). The correct classification rate was 71% in men and 63% in women (P=.133).
Referral Patterns
During the study period, 1714 women and 1965 men underwent an exercise echocardiographic study. Of these, 96 women and 244 men underwent coronary angiography. Nineteen percent of the women (70 of 366) with a positive exercise echocardiographic study were referred to angiography versus 27% of men (179 of 657) (P=.004). When the exercise echocardiographic study was negative, 2% of the women (26 of 1348) and 5% of the men (65 of 1308) were referred to angiography (P=.0001).
Adjustment for Verification Bias
Of the variables considered, age, sex, typical angina, diabetes, positive exercise ECG, and positive exercise echocardiography were independently related to the presence of CAD through the use of logistic regression analysis (Table 5). The Figure displays the observed and predicted probabilities of disease in the angiographic group. Table 6 shows the clinical and exercise echocardiographic data in 1618 women and 1721 men in the nonangiographic group. The probabilities of disease calculated through application of the logistic regression model to the nonangiographic group were summed separately according to exercise echocardiography results and added to the corresponding observed frequencies in the angiographic subset. The debiased sensitivity was 32% in women (95% CI, 25% to 40%) and 42% in men (95% CI, 38% to 49%). The debiased specificity was 86% in women (95% CI, 82% to 89%) and 83% in men (95% CI, 77% to 87%). The difference in adjusted sensitivity between men and women was significant at 11% (95% CI, 6% to 15%), whereas there was no difference in specificity (-2.9%; 95% CI, -7.0% to 0.7%). Calculations were made for several estimates of CAD prevalence (Table 7). The adjusted specificities were higher in men, whereas the sensitivity estimates were lower in women.
|
|
|
|
To determine the effect of the extent of CAD on the adjusted sensitivity, angiographically based multivariable models were used to predict the extent of CAD (single-, two-, or three-vessel disease) in the whole population, who underwent exercise echocardiography. These calculations yielded sensitivity estimates (for single-, two-, and three- vessel disease) of 30%, 44%, and 56% in men and 24%, 31%, and 43% in women, respectively. The corresponding debiased sensitivities were for men and women combined 27% for single-vessel disease (95% CI, 25% to 31%), 39% for two-vessel disease (95% CI, 34% to 46%), and 52% for three-vessel disease (95% CI, 44% to 62%). With adjustment for the differences in extent of disease in men and women, the difference in sensitivity attributable to difference in extent of disease was 3%. Thus, of the 11% difference in overall sensitivity between men and women, only 3% was attributable to differences in extent of CAD.
The simplified method by Diamond28 was also used to correct for test verification bias (Table 8). The results obtained with the two methods are comparable, which implies that bias is mainly the result of the preferential referral of positive tests to angiography.
|
|
Discussion |
|---|
|
TopAbstractIntroductionMethodsResultsDiscussionReferences |
|---|
These results document two major points. First, in practice, the observed sensitivity and specificity of exercise echocardiography differ from that initially reported.1 2 3 4 In the present center, exercise echocardiography was implemented at the time when most validation studies were published,1 2 3 hence it became rapidly used as part of the decision-making process to refer patients to angiography, as illustrated by the referral patterns according to the results of the test (Table 5
). The effect of this practice is to modify the specificity and sensitivity of the test as illustrated by the results before and after adjustment for test referral bias. A similar effect of test referral bias on other noninvasive stress tests modalities had been reported, but this had not been studied for exercise echocardiography5 6 7 29 30 (Table 9). While the different stress tests listed in Table 8 cannot be directly compared because of differences in methods and populations between studies, one can observe that the sensitivity of noninvasive tests, once corrected for referral bias, is generally low.
|
The second point is the difference between sexes, as shown by a lower positive predictive value and a lower adjusted sensitivity in women. Assessment of the value of any stress test modality in women compared with men is a complex issue because the observed results reflect the combined effect of intrinsic test performance, prevalence, and severity of disease and referral pattern; the last three parameters differ between men and women, as shown by the present data and previous studies,8 16 17 and should be considered when interpreting results. The lower positive predictive values in women can be expected from the lower prevalence of CAD in women, in agreement with Bayes theorem.9 This had been shown for exercise ECG8 10 11 12 but not for exercise echocardiography. Regarding sensitivity and specificity, the observed values were similar in both sexes but with markedly different referral patterns and thus not interpretable without taking referral into account.5 7 26 Indeed, further analysis adjusting for referral bias showed that sensitivity, a parameter that is not affected by prevalence, is lower in women than in men; this appears only partially explained by the difference in the severity of CAD between men and women. This raises the question of an "intrinsically" lower sensitivity in women, which deserves several comments. Despite the fact that it is not widely recognized, the concept of a lower sensitivity of exercise testing in women is not new. Hlatky and colleagues31 reported a markedly lower sensitivity of exercise ECG in women independent of the extent of atherosclerosis; several reports emphasized the lower sensitivity of exercise ECG or dipyridamole thallium in women compared with men32 33 and others indicated an increased frequency of false-negative results of dobutamine stress echocardiography and scintigraphy in women compared with men.34 While there is no explanation for this difference, it is conceivable, although speculative, that "qualitative" aspects, ie, physiological consequence of CAD, may differ between men and women. The implications of such a difference in the sensitivity of noninvasive tests between men and women in terms of outcome and cost-effective utilization of tests are beyond the scope of this study but clearly deserve further investigation. Such a finding underscores the importance of considering both sexes while analyzing the results of stress tests. Previous studies on stress echocardiography that included only women could not, by design, show such a difference.18 19 20 In addition, more than half of the women in these angiographic series had negative tests. This is in contrast to the frequency of a negative exercise echocardiographic study in the angiographic group of the present series, which was only 27%. This difference in frequency of negative tests between studies documents different referral patterns that need to be considered in the interpretation of the results of any stress test, as the present study documents.
Study Limitations
Logistic regression analysis has limitations regarding disease prediction35 ; this implies that the estimates of adjusted sensitivity and specificity reported herein should be considered as indicative of trends rather than absolute numbers. Even though the model used in the present study was not validated in a different group, the variables are consistent with previously published data.36 Most of the disease risk factors identified in men are also important in women, but the magnitude of their effect may differ37 38 and hence their use to predict CAD through the use of logistic regression analysis may be more problematic; this may in turn affect the estimates of adjusted sensitivity.
The interval of 1 year between exercise echocardiography and angiography was chosen to have the largest possible angiographic group in order to optimize the strength of the model to predict CAD. This interval may be perceived as long, but it has been used in a previously published study addressing a similar issue20 ; besides, in the present study, 92% of the angiographies were carried out within 6 months of the exercise echocardiography, and the results were the same for both time frames.
The exercise echocardiographic studies were interpreted by experienced echocardiographers, and intraobserver and interobserver variabilities were not assessed as part of this study. This issue has been addressed in other studies,39 40 and the purpose of the present study was to examine the performance of the test as it is being used in clinical practice.
Conventional angiographic reading has limitations regarding the physiological significance and the true extent of CAD.41 However, electronic calipers, which were chosen as the reference method in ongoing randomized trials42 and were shown to be a suitable alternative to quantitative coronary angiography,43 were used independently in a subset of this series; it gave results similar to visual assessment, allowing the use of conventional reading for the remainder of the analysis. Conventional reading reflects best clinical practice, since it remains the most widely used method to interpret coronary angiography; it is also the method used in studies for clinical decision making.44 However, the known limitations of angiography, which assesses anatomy, to determine the physiological consequences of coronary lesions, should be remembered when interpreting the "low" sensitivity of noninvasive diagnostic tests that is observed once correction for referral bias is performed (Table 9).
Recent studies have generated controversy over the management of women with known or suspected CAD.15 16 17 45 46 47 While the present study documents differential referrals to angiography between men and women, it was not designed to address appropriateness of care.
Conclusions
These results illustrate how exercise echocardiography can be expected to perform in clinical practice. Both the observed and adjusted sensitivities and specificities are similar to those reported for other imaging stress test modalities when used for patient management.
In women, the test appears to have a lower yield compared with that of men, since the positive predictive value and adjusted sensitivity were lower in women; the observation of a lower positive predictive value is consistent with the prevalence of CAD in this group. The lower estimated sensitivity is more disturbing and is congruent with other studies that have shown a lower sensitivity for other stress test modalities in women. This indicates the crucial importance of conducting further studies of the diagnostic and prognostic value of stress testing in women compared with men in clinical practice, in order to optimize its use at a time when the importance of cost-effectiveness is increasingly recognized.
Received July 29, 1996; revision received August 26, 1996; accepted August 31, 1996.
|
References |
|---|
|
TopAbstractIntroductionMethodsResultsDiscussionReferences |
|---|
1. Armstrong WF, O'Donnell J, Dillon JC, McHenry PL, Morris SN, Feigenbaum H. Complementary value of two-dimensional exercise echocardiography to routine treadmill exercise testing. Ann Intern Med. 1986;105:829-835.
2. Ryan T, Vasey CG, Presti CF, O'Donnell JA, Feigenbaum H, Armstrong WF. Exercise echocardiography: detection of coronary artery disease in patients with normal left ventricular wall motion at rest. J Am Coll Cardiol. 1988;11:993-999.[Abstract]
3. Crouse LJ, Harbrecht JJ, Vacek JL, Rosamond TL, Kramer PH. Exercise echocardiography as a screening test for coronary artery disease and correlation of coronary arteriography. Am J Cardiol. 1991;67:1213-1218.[Medline] [Order article via Infotrieve]
4. Marwick TH, Nemec JJ, Pashkow FJ, Stewart WJ, Salcedo EE. Accuracy and limitations of exercise echocardiography in a routine clinical setting. J Am Coll Cardiol. 1992;19:74-81.[Abstract]
5. Rozanski A, Diamond GA, Berman D, Forrester JS, Morris D, Swan HJC. The declining specificity of exercise radionuclide ventriculography. N Engl J Med. 1983;309:518-522.[Abstract]
6. Diamond GA. How accurate is SPECT thallium scintigraphy? J Am Coll Cardiol. 1990;16:1017-1021.[Medline] [Order article via Infotrieve]
7. Diamond GA. Reverend Bayes' silent majority: an alternative factor affecting sensitivity and specificity of exercise electrocardiography. Am J Cardiol. 1986;57:1175-1180.[Medline] [Order article via Infotrieve]
8. Weiner DA, Ryan TJ, McCabe CH, Kennedy JW, Schloss M, Tristani F, Chaitman BR, Fisher LD. Exercise stress testing: correlations among history of angina, ST segment response and prevalence of coronary artery disease in the coronary artery surgery study (CASS). N Engl J Med. 1979;301:230-235.[Abstract]
9. Diamond GA, Forrester JS. Analysis of probability as an aid in the clinical diagnosis of coronary artery disease. N Engl J Med. 1979;300:1350-1358.[Abstract]
10. Sketch MH, Mohiuddin SM, Lynch JD, Zencka AE, Runco V. Significant sex differences in the correlation of electrocardiographic exercise testing and coronary arteriograms. Am J Cardiol. 1975;36:169-172.[Medline] [Order article via Infotrieve]
11.
Barolsky SM, Gilbert CA, Faruqui A, Nutter DO, Schlant RC. Differences in electrocardiographic response to exercise of women and men: a non-Bayesian factor. Circulation. 1979;60:1021-1027.
12. Tavel ME. Specificity of electrocardiographic stress test in women versus men. Am J Cardiol. 1992;70:545-547.[Medline] [Order article via Infotrieve]
13.
Higginbotham MB, Morris KG, Coleman RE, Cobb FR. Sex-related differences in the normal cardiac response to upright exercise. Circulation. 1984;70:357-366.
14. Hanley PC, Zinsmeister AR, Clements IP, Bove AA, Brown ML, Gibbons RJ. Gender-related differences in cardiac response to supine exercise assessed by radionuclide angiography. J Am Coll Cardiol. 1989;13:624-629.[Abstract]
15. Tobin JN, Wassertheil-Smoller S, Wexler JP, Steingart RM, Budner N, Lense L, Wachspress J. Sex bias in considering coronary bypass surgery. Ann Intern Med. 1987;107:19-25.
16. Ayanian JZ, Epstein AM. Differences in the use of procedures between women and men hospitalized for coronary artery disease. N Engl J Med. 1991;325:221-225.[Abstract]
17.
Shaw LJ, Miller DD, Romeis JC, Kargl D, Younis LT, Chaitman BR. Gender differences in the noninvasive evaluation and management of patients with suspected coronary artery disease. Ann Intern Med. 1994;120:559-566.
18. Sawada SG, Ryan T, Fineberg NS, Armstrong WF, Judson WE, McHenry PL, Feigenbaum H. Exercise echocardiographic detection of coronary artery disease in women. J Am Coll Cardiol. 1989;14:1440-1447.[Abstract]
19. Williams MJ, Marwick TH, O'Gorman D, Foale RA. Comparison of exercise echocardiography with an exercise score to diagnose coronary artery disease in women. Am J Cardiol. 1994;74:435-438.[Medline] [Order article via Infotrieve]
20. Marwick TH, Anderson T, Williams MJ, Haluska B, Melin JA, Pashkow F, Thomas JD. Exercise echocardiography is an accurate and cost efficient technique for detection of coronary artery disease in women. J Am Coll Cardiol. 1995;26:335-341.[Abstract]
21. Feigenbaum H. Exercise echocardiography. J Am Soc Echocardiogr. 1988;1:161-166.[Medline] [Order article via Infotrieve]
22. Roger VL, Pellikka PA, Oh JK, Miller FA, Seward JB, Tajik AJ. Stress echocardiography, I: exercise echocardiography: techniques, implementation, clinical applications and correlations. Mayo Clin Proc. 1995;70:5-15.[Abstract]
23. Schiller NB, Shah PM, Crawford M, DeMaria A, Devereux R, Feigenbaum H, Gutgesell H, Reichek N, Sahn D, Schnittger I, Silverman NH, Tajik AJ. Recommendations for quantification of the left ventricle by two-dimensional echocardiography: American Society of Echocardiography Committee on Standards: Subcommittee on Quantification of Two-Dimensional Echocardiograms. J Am Soc Echocardiogr. 1989;2:358-367.[Medline] [Order article via Infotrieve]
24.
Quinones MA, Pickering E, Alexander JK. Percentage of shortening of the echocardiographic left ventricular dimension: its use in determining ejection fraction and stroke volume. Chest. 1978;74:59-65.
25. Rich S, Sheikh A, Gallastegui J, Kondos GT, Mason T, Lam W. Determination of left ventricular ejection fraction by visual estimation during real-time two-dimensional echocardiography. Am Heart J. 1982;104:603-606.[Medline] [Order article via Infotrieve]
26. Begg CB, Greenes RA. Assessment of diagnostic tests when disease verification is subject to selection bias. Biometrics. 1983;39:207-215.[Medline] [Order article via Infotrieve]
27. Efron B, Tibshirani R. Bootstrap methods for standard errors, confidence intervals and other measures of statistical accuracy. Stat Sci. 1986;1:54-77.
28. Diamond GA. Affirmative actions: can the discriminant accuracy of a test be determined in the face of selection bias? Med Decis Making. 1991;11:48-56.
29. Morise AP, Diamond GA. A comparison of the sensitivity and specificity of exercise electrocardiography in men and women using biased and unbiased populations of men and women. Am Heart J. 1995;130:741-747.[Medline] [Order article via Infotrieve]
30.
Schwartz RS, Jackson WG, Celio PV, Richardson LA, Hickman JR Jr. Accuracy of exercise 201thallium myocardial scintigraphy in asymptomatic young men. Circulation. 1993;87:165-172.
31. Hlatky MA, Pryor DB, Harrell FE Jr, Califf RM, Mark DB, Rosati RA. Factors affecting sensitivity and specificity of exercise electrocardiography: multivariable analysis. Am J Med. 1984;77:64-71.[Medline] [Order article via Infotrieve]
32. Gibbons RJ. Exercise ECG testing with and without radionuclide studies. In: Wenger NK, Speroff L, Packard B, eds. Cardiovascular Health and Disease in Women. Proceedings of an NHLBI Conference. Greenwich, Conn: Le Jacq Communications, Inc; 1993:73-80.
33. Kong BA, Shaw L, Miller DD, Chaitman BR. Comparison of accuracy for detecting coronary artery disease and side-effect profile of dipyridamole thallium-201 myocardial perfusion imaging in women versus men. Am J Cardiol. 1992;70:168-173.[Medline] [Order article via Infotrieve]
34. Marwick T, D'Hondt A, Baudhuin T, Willemart B, Wijns W, Detry JM, Melin J. Optimal use of dobutamine stress for the detection and evaluation of coronary artery disease: combination with echocardiography or scintigraphy or both? J Am Coll Cardiol. 1993;22:159-167.[Abstract]
35. Diamond GA. Future imperfect. The limitations of clinical prediction models and the limits of clinical prediction. J Am Coll Cardiol. 1989;14(suppl A):12A-22A.
36. Morise AP, Detrano R, Bobbio M, Diamond GA. Development and validation of a logistic regression-derived algorithm for estimating the incremental probability of coronary disease before and after exercise testing. J Am Coll Cardiol. 1992;20:1187-1196.[Abstract]
37. Isles CG, Hole DJ, Hawthorne VM, Lever AF. Relation between coronary risk and coronary mortality in women of the Renfrew and Paisley survey: comparison with men. Lancet. 1992;339:702-706.[Medline] [Order article via Infotrieve]
38.
Rich-Edwards JW, Manson JE, Hennekens CH, Buring JE. The primary prevention of coronary heart disease in women. N Engl J Med. 1995;332:1758-1766.
39. Oberman A, Fan PH, Nanda NC, Lee JY, Huster WJ, Sulentic JA, Storey OF. Reproducibility of two dimensional exercise echocardiography. J Am Coll Cardiol. 1989;14:923-928.[Abstract]
40. Hoffmann R, Lethen H, Marwick T, Arnese M, Fioretti P, Pingitore A, Picano E, Buck T, Erbel R, Flachskampf FA, Hanrath P. Analysis of interinstitutional observer agreement in interpretation of dobutamine stress echocardiograms. J Am Coll Cardiol. 1996;27:330-336.[Abstract]
41. White CW, Wright CB, Doty DB, Hiratza LF, Eastham CL, Harrison DG, Marcus ML. Does visual interpretation of the coronary arteriogram predict the physiologic importance of a coronary stenosis? N Engl J Med. 1984;310:819-824.[Abstract]
42. BARI Investigators. Protocol for the bypass angioplasty revascularization investigation. Circulation. 1991;84(suppl V):V-1-V-27.
43.
Uehata A, Matsuguchi T, Bittl JA, Orav J, Meredith IT, Anderson TJ, Selwyn AP, Ganz P, Yeung AC. Accuracy of electronic digital calipers compared with quantitative angiography in measuring coronary arterial diameter. Circulation. 1993;88:1724-1729.
44.
Coronary Artery Surgery Study (CASS). A randomized trial of coronary artery bypass surgery: survival data. Circulation. 1983;68:939-950.
45. Wenger NK. Gender, coronary artery disease, and coronary bypass surgery. Ann Intern Med. 1990;112:557-558.
46. Krumholz HM, Douglas PS, Lauer MS, Pasternak RC. Selection of patients for coronary angiography and coronary revascularization early after myocardial infarction: is there evidence for a gender bias? Ann Intern Med. 1992;116:785-790.
47.
Douglas PS. Gender, cardiology, and optimal medical care. Circulation. 1986;74:917-919.
This article has been cited by other articles:
 |
|
 |
|
  M. Jelinek Spectrum bias: why generalists and specialists do not connect Evid. Based Med., October 1, 2008; 13(5): 132 - 133. [Full Text] [PDF]
|
|
|
|
 |
|
 R. Sicari, P. Nihoyannopoulos, A. Evangelista, J. Kasprzak, P. Lancellotti, D. Poldermans, J.-U. Voigt, J. L. Zamorano, and on behalf of the European Association of Echocardi Stress echocardiography expert consensus statement: European Association of Echocardiography (EAE) (a registered branch of the ESC) Eur J Echocardiogr, July 1, 2008; 9(4): 415 - 437. [Abstract] [Full Text] [PDF]
|
|
|
|
 |
|
 R. Hachamovitch and M. F. Di Carli Methods and Limitations of Assessing New Noninvasive Tests: Part I: Anatomy-Based Validation of Noninvasive Testing Circulation, May 20, 2008; 117(20): 2684 - 2690. [Full Text] [PDF]
|
|
|
|
 |
|
 V. Stangl, V. Witzel, G. Baumann, and K. Stangl Current diagnostic concepts to detect coronary artery disease in women Eur. Heart J., March 2, 2008; 29(6): 707 - 717. [Abstract] [Full Text] [PDF]
|
|
|
|
 |
|
 J. C. Plana, I. A. Mikati, H. Dokainish, N. Lakkis, J. Abukhalil, R. Davis, B. C. Hetzell, and W. A. Zoghbi A randomized cross-over study for evaluation of the effect of image optimization with contrast on the diagnostic accuracy of dobutamine echocardiography in coronary artery disease The OPTIMIZE Trial. J. Am. Coll. Cardiol. Img., March 1, 2008; 1(2): 145 - 152. [Abstract] [Full Text] [PDF]
|
|
|
|
 |
|
 M. S. Lauer, S. C. Murthy, E. H. Blackstone, I. C. Okereke, and T. W. Rice [18F]Fluorodeoxyglucose Uptake by Positron Emission Tomography for Diagnosis of Suspected Lung Cancer: Impact of Verification Bias Arch Intern Med, January 22, 2007; 167(2): 161 - 165. [Abstract] [Full Text] [PDF]
|
|
|
|
|
|
P. Kligfield and M. S. Lauer Exercise Electrocardiogram Testing: Beyond the ST Segment Circulation, November 7, 2006; 114(19): 2070 - 2082. [Full Text] [PDF]
|
|
|
|
|
|
A. R. Albers, M. Z. Krichavsky, and G. J. Balady Stress Testing in Patients With Diabetes Mellitus: Diagnostic and Prognostic Value Circulation, January 31, 2006; 113(4): 583 - 592. [Full Text] [PDF]
|
|
|
|
 |
|
 W. F. Armstrong and W. A. Zoghbi Stress Echocardiography: Current Methodology and Clinical Applications J. Am. Coll. Cardiol., June 7, 2005; 45(11): 1739 - 1747. [Abstract] [Full Text] [PDF]
|
|
|
|
 |
|
 Y. Hayashino, M. Goto, Y. Noguchi, and T. Fukui Ventilation-Perfusion Scanning and Helical CT in Suspected Pulmonary Embolism: Meta-Analysis of Diagnostic Performance Radiology, March 1, 2005; 234(3): 740 - 748. [Abstract] [Full Text] [PDF]
|
|
|
|
|
|
J. H. Mieres, L. J. Shaw, A. Arai, M. J. Budoff, S. D. Flamm, W. G. Hundley, T. H. Marwick, L. Mosca, A. R. Patel, M. A. Quinones, et al. Role of Noninvasive Testing in the Clinical Evaluation of Women With Suspected Coronary Artery Disease: Consensus Statement From the Cardiac Imaging Committee, Council on Clinical Cardiology, and the Cardiovascular Imaging and Intervention Committee, Council on Cardiovascular Radiology and Intervention, American Heart Association Circulation, February 8, 2005; 111(5): 682 - 696. [Abstract] [Full Text] [PDF]
|
|
|
|
 |
|
 E. H. Blackstone and M. S. Lauer Caveat emptor: The treachery of work-up bias J. Thorac. Cardiovasc. Surg., September 1, 2004; 128(3): 341 - 344. [Full Text] [PDF]
|
|
|
|
|
|
K. Nasir, R. F. Redberg, M. J. Budoff, E. Hui, W. S. Post, and R. S. Blumenthal Utility of Stress Testing and Coronary Calcification Measurement for Detection of Coronary Artery Disease in Women Arch Intern Med, August 9, 2004; 164(15): 1610 - 1620. [Abstract] [Full Text] [PDF]
|
|
|
|
 |
|
 P. Whiting, A. W.S. Rutjes, J. B. Reitsma, A. S. Glas, P. M.M. Bossuyt, and J. Kleijnen Sources of Variation and Bias in Studies of Diagnostic Accuracy: A Systematic Review Ann Intern Med, February 3, 2004; 140(3): 189 - 202. [Abstract] [Full Text] [PDF]
|
|
|
|
|
|
M. D. Cheitlin, W. F. Armstrong, G. P. Aurigemma, G. A. Beller, F. Z. Bierman, J. L. Davis, P. S. Douglas, D. P. Faxon, L. D. Gillam, T. R. Kimball, et al. ACC/AHA/ASE 2003 guideline update for the clinical application of echocardiography: summary article: a report of the American college of cardiology/American heart association task force on practice guidelines (ACC/AHA/ASE committee to update the 1997 guidelines for the clinical application of echocardiography) J. Am. Coll. Cardiol., September 3, 2003; 42(5): 954 - 970. [Full Text] [PDF]
|
|
|
|
|
|
M. D. Cheitlin, W. F. Armstrong, G. P. Aurigemma, G. A. Beller, F. Z. Bierman, J. L. Davis, P. S. Douglas, D. P. Faxon, L. D. Gillam, T. R. Kimball, et al. ACC/AHA/ASE 2003 Guideline Update for the Clinical Application of Echocardiography: Summary Article: A Report of the American College of Cardiology/American Heart Association Task Force on Practice Guidelines (ACC/AHA/ASE Committee to Update the 1997 Guidelines for the Clinical Application of Echocardiography) Circulation, September 2, 2003; 108(9): 1146 - 1162. [Full Text] [PDF]
|
|
|
|
|
|
J Peteiro, L Monserrat, R Perez, E Vazquez, J.M Vazquez, and A Castro-Beiras Accuracy of peak treadmill exercise echocardiography to detect multivessel coronary artery disease: comparison with post-exercise echocardiography Eur J Echocardiogr, September 1, 2003; 4(3): 182 - 190. [Abstract] [Full Text] [PDF]
|
|
|
|
 |
|
 V. L. Roger, S. J. Jacobsen, S. A. Weston, P. A. Pellikka, T. D. Miller, K. R. Bailey, and B. J. Gersh Sex Differences in Evaluation and Outcome After Stress Testing Mayo Clin. Proc., July 1, 2002; 77(7): 638 - 645. [Abstract] [PDF]
|
|
|
|
|
|
A. M. Arruda-Olson, E. M. Juracan, D. W. Mahoney, R. B. McCully, V. L. Roger, and P. A. Pellikka Prognostic value of exercise echocardiographyin 5,798 patients: is there a gender difference? J. Am. Coll. Cardiol., February 20, 2002; 39(4): 625 - 631. [Abstract] [Full Text] [PDF]
|
|
|
|
 |
|
 N. K Wenger Clinical characteristics of coronary heart disease in women: emphasis on gender differences Cardiovasc Res, February 15, 2002; 53(3): 558 - 567. [Full Text] [PDF]
|
|
|
|
|
|
J.-W. Ha, E. M. Juracan, D. W. Mahoney, J. K. Oh, C. Shub, J. B. Seward, and P. A. Pellikka Hypertensive response to exercise: a potential cause for new wall motion abnormality in the absence of coronary artery disease J. Am. Coll. Cardiol., January 16, 2002; 39(2): 323 - 327. [Abstract] [Full Text] [PDF]
|
|
|
|
|
|
S. V. Williams, S. D. Fihn, and R. J. Gibbons Guidelines for the Management of Patients with Chronic Stable Angina: Diagnosis and Risk Stratification Ann Intern Med, October 2, 2001; 135(7): 530 - 547. [Abstract] [Full Text] [PDF]
|
|
|
|
 |
|
 A. P. Morise Are the American College of Cardiology/American Heart Association Guidelines for Exercise Testing for Suspected Coronary Artery Disease Correct? Chest, August 1, 2000; 118(2): 535 - 541. [Abstract] [Full Text] [PDF]
|
|
|
|
|
|
L. F. Bielak, J. A. Rumberger, P. F. Sheedy II, R. S. Schwartz, and P. A. Peyser Probabilistic Model for Prediction of Angiographically Defined Obstructive Coronary Artery Disease Using Electron Beam Computed Tomography Calcium Score Strata Circulation, July 25, 2000; 102(4): 380 - 385. [Abstract] [Full Text] [PDF]
|
|
|
|
 |
|
 R. J Gibbons IMAGING TECHNIQUES: Myocardial perfusion imaging Heart, March 1, 2000; 83(3): 355 - 360. [Full Text]
|
|
|
|
 |
|
 V. L. Roger, M. E. Farkouh, S. A. Weston, G. S. Reeder, S. J. Jacobsen, A. R. Zinsmeister, B. P. Yawn, S. L. Kopecky, and S. E. Gabriel Sex Differences in Evaluation and Outcome of Unstable Angina JAMA, February 2, 2000; 283(5): 646 - 652. [Abstract] [Full Text] [PDF]
|
|
|
|
|
|
C. Z. Dickinson, H. J. Dworkin, J. H. O'Keefe, Jr, T. M. Bateman, R. J. Gibbons, S. Kymes, L. Shaw, J. Fletcher, K. E. Fleischmann, M. G. M. Hunink, et al. Exercise Echocardiography vs Exercise SPECT Testing JAMA, November 3, 1999; 282(17): 1621 - 1623. [Full Text] [PDF]
|
|
|
|
|
|
R. J. Gibbons, K. Chatterjee, J. Daley, J. S. Douglas, S. D. Fihn, J. M. Gardin, M. A. Grunwald, D. Levy, B. W. Lytle, R. A. O'Rourke, et al. ACC/AHA/ACP-ASIM guidelines for the management of patients with chronic stable angina: A report of the American College of Cardiology/American Heart Association Task Force on Practice Guidelines (Committee on Management of Patients With Chronic Stable Angina) J. Am. Coll. Cardiol., June 1, 1999; 33(7): 2092 - 2197. [Full Text] [PDF]
|
|
|
|
|
|
K. E. Fleischmann, M. G. M. Hunink, K. M. Kuntz, and P. S. Douglas Exercise Echocardiography or Exercise SPECT Imaging?: A Meta-analysis of Diagnostic Test Performance JAMA, September 9, 1998; 280(10): 913 - 920. [Abstract] [Full Text] [PDF]
|
|
|
|
|
|
V. L. Roger, S. J. Jacobsen, P. A. Pellikka, T. D. Miller, K. R. Bailey, and B. J. Gersh Gender differences in use of stress testing and coronary heart disease mortality: a population-based study in Olmsted County, Minnesota J. Am. Coll. Cardiol., August 1, 1998; 32(2): 345 - 352. [Abstract] [Full Text] [PDF]
|
|
|
|
|
|
L. Mosca, J. E. Manson, S. E. Sutherland, R. D. Langer, T. Manolio, E. Barrett-Connor, and E. Barrett-Connor Cardiovascular Disease in Women : A Statement for Healthcare Professionals From the American Heart Association Circulation, October 7, 1997; 96(7): 2468 - 2482. [Full Text]
|
|
|
|
|
|
P. S. Douglas Is Noninvasive Testing for Coronary Artery Disease Accurate? Circulation, January 21, 1997; 95(2): 299 - 302. [Full Text]
|
|
|
|
|
|
M. J. Budoff, G. A. Diamond, P. Raggi, Y. Arad, A. D. Guerci, T. Q. Callister, and D. Berman Continuous Probabilistic Prediction of Angiographically Significant Coronary Artery Disease Using Electron Beam Tomography Circulation, April 16, 2002; 105(15): 1791 - 1796. [Abstract] [Full Text] [PDF]
|
|
| |||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||