(Circulation. 2000;101:e99.)
© 2000 American Heart Association, Inc.
Circulation Electronic Pages |
Qualification of the Concepts of Unqualified Success and Unmitigated Failure
British Heart Foundation Research Fellow
Robert Luff Foundation Research Fellow
Professor of Cardiology Royal Brompton Hospital, London, UK
 |
Introduction |
|---|
 Top Introduction ReferencesIntroduction References |
|---|
Mancini and Schulzer1 have developed advanced methods for calculation of the benefit accruing from treatment, with new formulas for the "chance of unqualified success" of a treatment. The primary formula is (p1-p2)[1-(q1-q2)], where the primary end-point rates in the control and treatment groups are p1 and p2, respectively, and the adverse event rates q2 and q1, respectively. We suggest this formula cannot represent a probability calculation. Consider a hypothetical situation in which treatment reduces end points from 0.9 to 0.1 while reducing the rate of the adverse events from 0.6 to 0.2. The "chance of unqualified success" is 0.8x1.4=1.12, exceeding 1. Other legitimate p and q probabilities between 0 and 1 produce results ranging from -2 to +2. This contrasts with the allowable range of 0 to 1 for orthodox measures of probability. The formula can therefore be doubted, even when it returns values between 0 and 1.
Perhaps the formula the authors seek is [(1-p1)(1-q2)]- [(1-p2)(1-q1)]. This is still not truly a probability (since it ranges from -1 to +1), but it does represent the change (attributable to treatment) in absolute risk of the combined end point of primary end point and/or adverse effect. The reciprocal of its absolute value gives the number needed to treat to change the number of combined end points by 1.
The authors appear to believe that the difference of 2 probabilities should itself be a probability. We argue that this is only true in general when the events of 1 probability are a subset of those in the other, which can never be the case when 2 arms of a trial are being compared. Imagine a study of men and women in equal numbers, a disease that causes strokes in 90% of men but none in women, and a treatment that prevents all the strokes in men but causes strokes in all women. What is the chance that a stroke is prevented in the treatment arm in general? We believe the authors would answer "-10%", but this is the absolute risk difference and not a probability. Surely the true probability of a stroke having been prevented is 40% (which lies alongside the 10% probability of no effect and 50% of stroke causation, making 100%).
This extreme example illustrates the principle that the true probability of events being prevented cannot be calculated from overall event rates alone but is potentially subject to serial revisions on further patient characterization. This weakness is not shared by traditional combined end-point rates.
|
References |
|---|
|
TopIntroductionReferencesIntroduction References |
|---|
1. Mancini GBJ, Schulzer M. Reporting risks and benefits of therapy by use of the concepts of unqualified success and unmitigated failure: applications to highly cited trials in cardiovascular medicine. Circulation.. 1999;99:377–383.
Response
Professor of Medicine and Statistics University of British Columbia, Vancouver, BC, Canada
Professor and Head, Department of Medicine University of British Columbia, Vancouver, BC, Canada
|
Introduction |
|---|
|
TopIntroductionReferencesIntroduction References |
|---|
Kessler argues that as the NNTUS is generally larger than the NNT, more adverse events can be expected in the NNTUS group. This is tautologically true but does not alter the meaning of the NNTUS concept.R1 R2 The NNTUS estimates the expected number of individuals who need to be treated to achieve an unqualified success in 1 individual. We do not understand Kessler’s claim that the NNHUF does not "compensate for the adverse events that occur in the proportion of the population who cannot benefit from treatment." The NNHUF is calculated from this proportion. If one knew a priori which patients would be "harmed never having had the possibility of benefit," it would be unethical to enroll them into a clinical trial. Kessler’s assertion that the majority of adverse events will emanate from patients who will fail to benefit from the treatment implies a dependence between treatment benefit and treatment-induced adverse events. We have dealt with this situation separately.R1 ) The claim that interpretation is confounded when the primary event and the adverse event are not comparable was addressed by the use of utility functions.R1 (378–379),R2 "Comparability" was established by determining dollar costs of the desired and untoward effects of therapy.
Francis and colleagues assert that our primary formula, (p1-p2)x(1-[q1-q2]), "cannot represent a probability calculation." They should note the mathematical constraints specified in the derivation of this formula: p1>p2 and q1>q2.R1 R2 Under these constraints, the formula calculates the product of 2 probabilities. These constraints reflect the common scenario of a treatment that reduces the primary event rate while increasing certain adverse events. But other situations may occur: (1) a poor treatment may increase the primary event rate, and (2) an exceptional treatment may reduce both the primary and adverse event rates. In case 1, the success rate of "control" over treatment can be calculated by reversing the roles of the 2 arms of the study. We applied this to data from the Cardiac Arrhythmia Suppression Trial.R3 In case 2, the rate of treatment-induced adverse events relative to control may in an extreme case be reduced to zero, and an "unqualified success" becomes synonymous with "treatment success," with rate estimated by (p1-p2)x1. The writers claim that the difference of 2 probabilities represents a probability only "when the events of 1 probability are a subset of those in the other" and that "this can never be the case when 2 arms of a trial are being compared." Thus, it is quite impossible to apply this idealized notion to the evaluation of treatment efficacy in a randomized clinical trial. Their example of an extreme gender-by-treatment interaction actually fits into case 1, and calculations can still be carried out as previously outlined. Finally, while the combined end-point rates may have merit in some contexts, our goal was to further develop the NNT concept, which is more clinically useful.
|
References |
|---|
|
TopIntroductionReferencesIntroduction References |
|---|
1. Mancini GBJ, Schulzer M. Reporting risks and benefits of therapy by use of the concepts of unqualified success and unmitigated failure: applications to highly cited trials in cardiovascular medicine. Circulation. 1999;99:377–383.
2.
Schulzer M, Mancini GBJ. "Unqualified success" and
"unmitigated failure": number-needed-to-treat concepts for
assessing treatment efficacy in the presence of treatment-induced
adverse events. Int J Epidemiol. 1996;25:704–712.
3. Echt DS, Liebson PR, Mitchell LB, Peters RW, Obias-Manno D, Barker AH, Arensberg D, Baker A, Friedman L, Greene HL, Huther ML, Richardson DW, and the CAST Investigators. Mortality and morbidity in patients receiving encainide, flecainide, or placebo: the Cardiac Arrhythmia Suppression Trial. N Engl J Med. 1991;324:1605–1611.[Abstract]
| |||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||