(Circulation. 1999;100:1971-1976.)
© 1999 American Heart Association, Inc.
Clinical Investigation and Reports |
From the Departments of Cardiology (S.G.E., V.G., P.L.W., E.J.T.) and Biostatistics (D.M.), The Cleveland Clinic Foundation, Cleveland, Ohio.
Correspondence to Stephen G. Ellis, MD, The Cleveland Clinic Foundation, 9500 Euclid Ave, F-25, Cleveland, OH 44195. E-mail elliss{at}cesmtp.ccf.org
| Abstract |
|---|
|
|
|---|
7% of patients. Major
advances in treatment options would suggest that this scheme may be
outmoded, but the schemes that have been suggested to update lesion
classification have not been widely accepted.
Methods and ResultsFour thousand one hundred eighty-one
consecutive patients (6676 lesions) formed a training set and 2146
patients (4231 lesions) formed a validation set treated from 1995 to
1997 at a single center used by 3 hospital groups. Twenty-seven
pretreatment candidate variables were analyzed with the use
of stepwise proportional logistic regression, and 9 (nonchronic total
occlusion with TIMI flow 0, degenerated vein graft, vein graft age >10
years, lesion length
10 mm, severe calcium, lesion irregularity,
large filling defect, angulated
45 degrees plus calcium, and
eccentricity) were independently correlated (P<0.05)
with ranked adverse outcome (death, Q-wave or creatine kinase
3x
normal myocardial infarction, or emergency coronary artery
bypass grafting>>creatine kinase 2 to 3x myocardial
infarction>>possibly related to nonQ-wave myocardial infarction>>no
complication). A scheme based on these findings and the old American
College of Cardiology/American Heart Association scheme
were found to have c-statistics in the validation set of 0.672 and
0.620 (P=0.010 vs old scheme), respectively.
ConclusionsAppreciation of these contemporary risk factors for complications of coronary intervention may assist in patient selection and in risk adjustment for comparison of outcomes between providers.
Key Words: angioplasty stents platelet aggregation inhibitors risk factors angiography
| Introduction |
|---|
|
|
|---|
Our purpose was to develop a new predictive model for adverse outcomes with the use of contemporary data and to compare its performance in a validation sample against that of the current modified ACC/AHA scheme.2
| Methods |
|---|
|
|
|---|
Variables and Definitions
Baseline Clinical Variables
Acute MI (<24 hours), age, cardiogenic shock, current smoking,
diabetes mellitus (insulin-dependent and noninsulin-dependent),
gender, hypercholesteremia (total cholesterol
240 mg%,
LDL cholesterol
130 mg%, or HDL cholesterol
<35 mg%), hypertension, New York Heart Association congestive heart
failure class, prior remote MI (>2 weeks), recent MI (1 to 14 days),
saphenous vein graft age (years), and surgically inoperable, unstable
angina. Except for vein graft age (analogous to "chronic" total
occlusion in the ACC/AHA scheme in that it relates a time duration with
a morphological or locating variable), these variables were
used for descriptive purposes only.
Angiographic Variables
See Table 1
for
variables assessed and their definitions. Angiographic
variables were analyzed by 1 of 2 independent observers
blinded to clinical outcome in the training set analysis and by
1 of 12 clinical interventionists overread by the Cleveland Clinic Core
Angiographic Laboratory in the validation set analysis.
|
Treatment Variables
Treatment variables were abciximab and primary treatment
device [balloon, directional atherectomy, laser, Rotablator, stent,
and transluminal extraction catheter (TEC)].
Outcome Variables
Outcome variables were cardiac death, emergency bypass
surgery (with acute ischemia or for the prevention of acute
ischemia within 24 hours of the procedure), and MI [Q-wave and
nonQ-wave/subcategorized by CK level (ECG changes or enzyme
elevations consequent to an acute MI at the patients
presentation are excluded)].
Statistical Analysis
Continuous data are presented as mean±1 SD if normally
distributed and as median and interquartile range if skewed.
Categorical data are presented as a percentage.
Cases for the training set analysis were composed of all patients treated during 1995 to 1996 with any ischemic complication and a randomly selected cohort of patients without complications (3:1 patients with no complications:patients with complications). This cohort was termed the enriched training set. This approach allowed us to study characteristics not routinely evaluated without having to review thousands of cineangiograms. However, if characteristics were relatively infrequent in this sample (n <50) and recorded routinely in our database, the entire 1995 to 1996 data set was used to ascertain their relation to clinical outcome. Secondary analyses of the population without the most predictive variables was performed to improve identification of intermediate-risk lesion morphologies. Cases for the validation set analysis were composed of all patients treated during 1997.
Adverse outcomes were characterized as most severe: cardiac death,
Q-wave MI, nonQ-wave MI with CK
5x upper limit of normal and
positive cardiac isoenzymes, or emergency bypass surgery; severe:
nonQ-wave MI with CK elevation 3 to 5x upper limit of normal in the
presence of abnormal cardiac isoenzymes; moderate: nonQ-wave MI with
CK elevation 2 to 3x upper limit of normal with positive cardiac
isoenzymes; uncertain: the presence of nonQ-wave MI, possibly
associated with the lesion evaluated (this typically occurred when
multiple lesions were treated and there was no obvious angiographic
complication); and no complication associated with lesion treatment.
Because of the small number of lesions associated with the severe
category in the test population, the most severe and severe categories
were subsequently collapsed for the purposes of all analyses
subsequent to our preliminary investigation.
Stepwise proportional odds logistic regression analysis12 was performed to identify independent correlates of adverse ischemic outcomes in the training sample. This, in contradistinction to standard multiple logistic regression, allows for simultaneous fitting of a model to a ranked ordinal outcome.
On the basis of their relation to graded adverse outcome, variables
were classified as strongly or moderately correlated. With the use of
the training sample, a scheme to optimize the true-positive,
false-negative relation was developed. The presence of any strong
correlate qualified the lesion as highest risk (class IV). The presence
of
3 moderate characteristics qualified the lesion as high risk
(class III), the presence of 1 to 2 moderate characteristics qualified
the lesion as moderate risk (class II), and absence of any risk factor
led to the assessment of low risk (class I).
The predictive capability of the new scheme and the current ACC/AHA scheme were then tested in the 1997 validation population. Comparison of their predictive capacities were performed with the use of the methods of Hanley and McNeil.13 Subgroup analyses were also performed in the 1997 stent-plus-abciximabtreated population.
All statistical analyses were performed with SYSTAT (SYSTAT version 7.0, SPSS Inc) or SAS/STAT (SAS Institute Inc, version 6.1).
| Results |
|---|
|
|
|---|
|
Clinical Outcomes
For the training and validation cohorts together, technical
success achieved at
1 target sites was achieved in 95.8% of
patients, and major complications (death, Q-wave MI, or emergency
bypass surgery) occurred in 1.9% of patients (Table 3
). There was no significant difference
between the training and validation sets for any major outcome.
|
Training Sample Evaluation
In the enriched training sample, there were 85 lesions associated
with "most severe" complications, 11 lesions associated with
"severe" complications, 93 lesions associated with "moderately
severe" complications, and 57 lesions associated with "possible"
complications (a total of 246 adverse eventrelated lesions). This
number increased to 257 in the full evaluation consequent to our
inability to retrieve cineangiograms on 7 patients (11
lesions) associated with complications during the analysis of
the enriched training sample.
As can be seen in Table 4
, 10
variables were identified as independent correlates of graded
adverse outcome. Detailed clinical outcomes for the 2 most important
correlates in the entire 1995 to 1997 cohort were for nonchronic total
occlusion: death, 4.5%; Q-wave infarction, 0.1%; emergency bypass
surgery, 2.8%; nonQ-wave infarction, 5.2% (CK >5x 0.6%, CK 3 to
5x 3.9%, CK 2 to 3x 0.7%); for degenerated saphenous vein grafts:
death, 1.1%; Q-wave infarction, 2.8%; emergency bypass surgery,
0.4%; nonQ-wave infarction, 12.7% (CK >5x 1.5%, CK 3 to 5x
8.8%, CK 2 to 3x 2.4%). The proposed schema derived from this
analysis is presented in Figure 1
(the 2 lesion length variables were
combined). The c-statistic evaluating the predictive value of the new
model for complications in the training set was 0.701.
|
|
Validation Sample Evaluation
Performance of the new model in the 1997 validation sample
is shown in Table 5
and Figure 2
. The calibration coefficient
(r) for the relation between risk in the training and
validation samples for each risk group (Figure 2
) is 0.971. The
c-statistic for the new model was 0.672 (the c-statistic for the
ACC/AHA scheme was 0.620, P=0.010). The new model had its
predictive accuracy maintained in the prespecified stent+abciximab
subset (c-statistic=0.663), whereas by the ACC/AHA scheme, the
c-statistic was only 0.589.
|
|
| Discussion |
|---|
|
|
|---|
Our comprehensive reevaluation of the relation between lesion morphology and short-term outcome with percutaneous intervention was undertaken with several goals and considerations. First, we wanted to develop and validate a better predictive instrument suitable for the current era of intervention. We expected that changes in treatment and outcome might attenuate the correlation of some previously important lesion characteristics with outcome. We intended to make use of inferences from observations with the use of intracoronary ultrasound and angioscopy17 18 19 to postulate new or modified angiographic characteristics to evaluate. We recognized, however, that prediction of risk generally becomes more difficult as overall risk diminishes (as it has in the decade since the modified ACC/AHA scheme was developed). Second, we hoped to more objectively characterize certain angiographic risk factors to lessen the subjectivity of the risk assessment. Third, we wanted for the first time to have a system that reflected the gradation of clinically meaningful adverse events.20 Fourth, we hoped that the new instrument would be relatively simple and easy to implement.
The proposed classification scheme is, in fact, more predictive than the prior modified ACC/AHA lesion scheme, although perhaps not by as much as one might have suspected beforehand. Importantly, it was validated in a patient population reflecting current utilization practice (stenting in 64%, glycoprotein IIb/IIIa inhibitors in 41%), and maintained its predictive accuracy in the patient populations treated with stents and ReoPro (c-statistic=0.66). In addition, it is somewhat simpler than the previous system analyzing 9 specific elements instead of 16. Further, we were able to more specifically define the nature of certain characteristics evaluated. Previous definitions that were particularly troublesome due to their subjectivity included that for degenerated vein grafts, calcification, and proximal tortuosity.
Also important, particularly as it pertains to the assessment of individual patient risk, is the absence of importance of certain characteristics that were previously thought to be associated with heightened risk. These include lesion angulation per se, bifurcation location, ostial location, proximal tortuosity, and small thrombus. It is likely that improvements in device profile and trackability, as well as the capacity for stenting to treat dissections that were particularly prone to develop with balloon angioplasty in certain circumstances, account for these differences.
In addition, the identification of specific lesion characteristics that
continue to be associated with complications, even with current
therapies, provides objective data from which to challenge industry to
improve percutaneous treatment devices and adjunct
pharmacology. The importance of aged and degenerated vein graft lesion
morphology in particular should heighten interest in the development of
treatments that may minimize the risk of complications associated with
treatment of such lesions, for example, emboli entrapment devices and
covered stents. The association of nonchronic total occlusion with
adverse outcome is partly due to its correlation with the setting of
acute MI and the subsequent risk of cardiac death; hence it may be less
modifiable. Nonetheless, if patients with acute MI were excluded from
analysis, this variable maintains considerable independent
prognostic power (OR=3.60, P=0.003 [compare with Table 4
]).
There are 2 limitations in particular that should be noted in relation to this analysis. First, although the model was prospectively validated in a population representative of currently treated patients, one must be somewhat cautious about overgeneralizing the results. The results were derived from outcomes of 18 generally experienced interventionists operating in an environment when senior operator assistance was readily available. Annual interventional patient volume of these operators ranged from 42 to 323 over this 3-year period. In addition, the assessment of lesion morphology was performed by angiographers who were trained and highly experienced. Second, although from a statistical standpoint this model provides considerable predictive power, it is far from perfect. Certainly, clinical factors such as patient age, presentation with acute MI, or poor left ventricular function, as well as operator characteristics,21 22 23 even in the era of stent utilization, would be expected to affect outcome and therefore diminish the sole predictive capability of lesion morphology. In the end, any measure of the relation between lesion characteristics and outcome in this setting, although somewhat helpful in the choice of potential therapies for a given patient, will always have greater power in assessing the risk of large populations.
In conclusion, utilization of this contemporary risk assessment algorithm by interventionists working under similar circumstances should improve their assessment of patient risk, and, in particular, extend availability of percutaneous intervention to some patients who otherwise would have been turned down for treatment because of perceived high risk.
| Acknowledgments |
|---|
Received April 21, 1999; revision received June 15, 1999; accepted July 13, 1999.
| References |
|---|
|
|
|---|
2.
Ellis SG, Vandormael MG, Cowley MJ, DiSciascio G,
Deligonul U, Topol EJ, Bulle TM. Coronary morphologic and
clinical determinants of procedural outcome with angioplasty for
multivessel coronary disease: implications for patient
selection. Circulation. 1990;82:11931202.
3.
Baim DS, Kent KM, King SB, Safian RD, Cowley MJ,
Holmes DR, Roubin GS, Gallup D, Steenkiste AR, Detre K, for the NACI
Investigators. Evaluating new devices: acute (in-hospital) results from
the New Approaches to Coronary Intervention Registry.
Circulation. 1994;89:471481.
4.
Reifart N, Vandormael M, Krajcar M, Göhring S,
Preusler W, Schwarz F, Störger H, Hofmann M, Klöpper J,
Müller S Haase J. Randomized comparison of angioplasty of complex
coronary lesions at a single center: excimer laser, rotational
atherectomy, and balloon angioplasty comparison (ERBAC) Study.
Circulation. 1997;96:9198.
5. Park S-J, Park S-W, Hong M-K, Cheong S-S, Lee CW, Kim J-J, Hong MK, Mintz GS Leon MB. Stenting of unprotected left main coronary artery stenoses: immediate and late outcomes. J Am Coll Cardiol. 1997;31:3742.
6.
EPIC Investigators. Use of a monoclonal antibody
directed against the platelet glycoprotein IIb/IIIa
receptor in high-risk coronary angioplasty. N Engl
J Med. 1994;330:956961.
7.
Haude M, Erbel R, Hoepp HW, Heublein B, Sigmund M,
Meyer J, and the STENT-BY Study Group. STENT-BY Study: a prospective
randomized trial comparing immediate stenting versus conservative
treatment strategies in abrupt vessel closure or
symptomatic dissections during coronary balloon
angioplasty. Eur Heart J. 1996;17:172. Abstract.
8. EPISTENT Investigators. Randomised placebo-controlled and balloon-angioplasty-controlled trial to assess safety of coronary stenting with use of platelet glycoprotein-IIb/IIIa blockade. Lancet. 1998;352:8792.[Medline] [Order article via Infotrieve]
9. Tan K, Sulke N, Taub N, Sowton E. Clinical and lesion morphologic determinants of coronary angioplasty success and complications: current experience. J Am Coll Cardiol. 1995;25:855865.[Abstract]
10. Kimmel SE, Berlin JA, Strom BL, Laskey WK, for the Registry Committee of the Society for Cardiac Angiography and Interventions. Development and validation of a simplified predictive index for major complications in contemporary percutaneous transluminal coronary angioplasty practice. J Am Coll Cardiol. 1995;26:931938.[Abstract]
11. Tenaglia AN, Fortin DF, Califf RM, Frid DJ, Nelson CL, Gardner L, Miller M, Navetta FI, Smith JE, Tcheng JE, Stack RS. Predicting the risk of abrupt vessel closure after angioplasty in an individual patient. J Am Coll Cardiol. 1994;24:10041011.[Abstract]
12. Special topics. Polytomous logistic regression: introduction to the model and estimation of the parameters. In: Hosmer DW, Lemeshow S, eds. Applied Logistic Regression. New York, NY: John Wiley & Sons; 1989:216245.
13.
Hanley JA, McNeil BJ. A method of comparing the areas
under receiver operating characteristic curves derived from the same
cases. Radiology. 1983;148:839843.
14. Myler RK, Shaw RE, Stertzer SH, Hecht HS, Ryan C, Rosenblum J, Cumberland DC, Murphy MC, Hansell HN, Hidalgo B. Lesion morphology and coronary angioplasty: current experience and analysis. J Am Coll Cardiol. 1992;19:16411652.[Abstract]
15. Hannan EL, Arani DT, Johnson LW, Kemp HG, Lukacik G. Percutaneous transluminal coronary angioplasty in New York State: risk factors and outcomes. JAMA. 1993;268:30923097.
16.
Ellis SG, Omoigui N, Bittl JA, Lincoff M, Wolfe MW,
Howell G, Topol EJ. Analysis and comparison of
operator-specific outcomes in interventional
cardiology: from a multicenter database of 4860
quality-controlled procedures. Circulation. 1996;93:431439.
17.
Fitzgerald PJ, Ports TA, Yock PG. Contribution of
localized calcium deposits to dissection after angioplasty: an
observational study using intravascular ultrasound.
Circulation. 1992;86:6470.
18.
Mintz GS, Popma JJ, Pichard AD, Kent KM, Satler LF,
Chuang YC, Ditrano CJ, Leon MB. Patterns of calcification in
coronary artery disease: a statistical analysis of
intravascular ultrasound and coronary angiography in 1155
lesions. Circulation. 1995;91:19591965.
19.
White CJ, Ramee SR, Collins TJ, Escobar AE, Karsan A,
Shaw D, Jain SP, Bass TA, Heuser RR, Teirstein PS, Bonan R, Walter PD,
Smalling RW. Coronary thrombi increase PTCA risk: angioscopy as
a clinical tool. Circulation. 1996;93:253258.
20.
Califf RM, Abdelmeguid AE, Kuntz RE, Popma JJ, Davidson
CJ, Cohen EA, Kleiman NS, Mahaffey KW, Topol EJ, Pepine CJ, Lipicky RJ,
Granger CB, Harrington RA, Tardiff BE, Crenshaw BS, Bauman RP,
Zuckerman BD, Chaitman BR, Bittl JA, Ohman EM. Myonecrosis after
revascularization procedures. J Am Coll
Cardiol. 1998;31:241251.
21. Hannan EL, Racz M, Ryan TJ, McCallister BD, Johnson LW, Arani DT, Guerci AD, Sosa J, Topol EJ. Coronary angioplasty volume-outcome relationships for hospitals and cardiologists. JAMA. 1997;279:892898.
22.
Ellis SG, Weintraub W, Holmes D, Shaw R, Block PC, King
SB. Relation of operator volume and experience to procedural outcome of
percutaneous coronary
revascularization at hospitals with high
interventional volumes. Circulation. 1997;95:24792484.
23.
Kastrati A, Neumann F-J, Schömig A. Operator
volume and outcome of patients undergoing coronary stent
placement. J Am Coll Cardiol. 1998;32:970976.
This article has been cited by other articles:
![]() |
E Romagnoli, F Burzotta, C Trani, M Siviglia, G G L Biondi-Zoccai, G Niccoli, A M Leone, I Porto, M A Mazzari, R Mongiardo, et al. EuroSCORE as predictor of in-hospital mortality after percutaneous coronary intervention Heart, January 1, 2009; 95(1): 43 - 48. [Abstract] [Full Text] [PDF] |
||||
![]() |
P. F Ludman Assessing the risks of percutaneous coronary intervention: do we have an equivalent of the EuroSCORE? Heart, November 1, 2008; 94(11): 1366 - 1369. [Full Text] [PDF] |
||||
![]() |
J. M. Wilson and J. T. Willerson Myocardial Revascularization with Percutaneous Devices Card. Surg. Adult, January 1, 2008; 3(2008): 573 - 598. [Full Text] |
||||
![]() |
S. R. Atmakuri, E. I. Lev, C. Alviar, E. Ibarra, A. E. Raizner, S. L. Solomon, and N. S. Kleiman Initial Experience With a Magnetic Navigation System for Percutaneous Coronary Intervention in Complex Coronary Artery Lesions J. Am. Coll. Cardiol., February 7, 2006; 47(3): 515 - 521. [Abstract] [Full Text] [PDF] |
||||
![]() |
P. Wenaweser, C. Rey, F. R. Eberli, M. Togni, D. Tuller, S. Locher, A. Remondino, C. Seiler, O. M. Hess, B. Meier, et al. Stent thrombosis following bare-metal stent implantation: success of emergency percutaneous coronary intervention and predictors of adverse outcome Eur. Heart J., June 2, 2005; 26(12): 1180 - 1187. [Abstract] [Full Text] [PDF] |
||||
![]() |
M. Singh, C. S. Rihal, R. J. Lennon, K. N. Garratt, and D. R. Holmes Jr Comparison of Mayo Clinic risk score and American College of Cardiology/American Heart Association lesion classification in the prediction of adverse cardiovascular outcome following percutaneous coronary interventions J. Am. Coll. Cardiol., July 21, 2004; 44(2): 357 - 361. [Abstract] [Full Text] [PDF] |
||||
![]() |
D. E. Cutlip, K. K. L. Ho, R. E. Kuntz, and D. S. Baim Risk assessment for percutaneous coronary intervention: our version of the weather report? J. Am. Coll. Cardiol., December 3, 2003; 42(11): 1896 - 1899. [Full Text] [PDF] |
||||
![]() |
M. Singh, C. S. Rihal, F. Selzer, K. E. Kip, K. Detre, and D. R. Holmes Jr Validation of Mayo clinic risk adjustment model for in-hospital complications after percutaneous coronary interventions, using the National Heart, Lung, and Blood Institute dynamic registry J. Am. Coll. Cardiol., November 19, 2003; 42(10): 1722 - 1728. [Abstract] [Full Text] [PDF] |
||||
![]() |
P. J. de Feyter and E. McFadden Risk score for percutaneous coronary intervention: forewarned is forearmed J. Am. Coll. Cardiol., November 19, 2003; 42(10): 1729 - 1730. [Full Text] [PDF] |
||||
![]() |
M. Singh, U. Rosenschein, K. K.L. Ho, P. B. Berger, R. Kuntz, and D. R. Holmes, Jr Treatment of Saphenous Vein Bypass Grafts With Ultrasound Thrombolysis: A Randomized Study (ATLAS) Circulation, May 13, 2003; 107(18): 2331 - 2336. [Abstract] [Full Text] [PDF] |
||||
![]() |
M. Singh, R. J. Lennon, D. R. Holmes Jr, M. R. Bell, and C. S. Rihal Correlates of procedural complicationsand a simple integer risk scorefor percutaneous coronary intervention J. Am. Coll. Cardiol., August 7, 2002; 40(3): 387 - 393. [Abstract] [Full Text] [PDF] |
||||
![]() |
S.J. Brener, S.G. Ellis, J. Schneider, and E.J. Topol Frequency and long-term impact of myonecrosis after coronary stenting Eur. Heart J., June 1, 2002; 23(11): 869 - 876. [Abstract] [Full Text] [PDF] |
||||
![]() |
R. E. Shaw, H. V. Anderson, R. G. Brindis, R. J. Krone, L. W. Klein, C. R. McKay, P. C. Block, L. J. Shaw, K. Hewitt, W. S. Weintraub, et al. Development of a risk adjustment mortality model using the American College of Cardiology-National Cardiovascular Data Registry (ACC-NCDR) experience: 1998-2000 J. Am. Coll. Cardiol., April 3, 2002; 39(7): 1104 - 1112. [Abstract] [Full Text] [PDF] |
||||
![]() |
S. G. Ellis, S. J. Brener, A. M. Lincoff, D. J. Moliterno, P. L. Whitlow, J. P. Schneider, and E. J. Topol {beta}-Blockers Before Percutaneous Coronary Intervention Do Not Attenuate Postprocedural Creatine Kinase Isoenzyme Rise Circulation, November 27, 2001; 104(22): 2685 - 2688. [Abstract] [Full Text] [PDF] |
||||
![]() |
D. P. Chew, D. L. Bhatt, M. A. Robbins, M. S. Penn, J. P. Schneider, M. S. Lauer, E. J. Topol, and S. G. Ellis Incremental Prognostic Value of Elevated Baseline C-Reactive Protein Among Established Markers of Risk in Percutaneous Coronary Intervention Circulation, August 28, 2001; 104(9): 992 - 997. [Abstract] [Full Text] [PDF] |
||||
![]() |
W. Maier, O. Mini, J. Antoni, M. B. Wischnewsky, and B. Meier ABC Stenosis Morphology Classification and Outcome of Coronary Angioplasty : Reassessment With Computing Techniques Circulation, March 6, 2001; 103(9): 1225 - 1231. [Abstract] [Full Text] [PDF] |
||||
| |||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
Circulation Home | Subscriptions | Archives | Feedback | Authors | Help | AHA Journals Home | Search Copyright © 1999 American Heart Association, Inc. All rights reserved. Unauthorized use prohibited. |