Validation of Risk Prediction Models to Detect Asymptomatic Carotid Stenosis

Abstract

Background Significant asymptomatic carotid stenosis (ACS) is associated with higher risk of strokes. While the prevalence of moderate and severe ACS is low in the general population, prediction models may allow identification of individuals at increased risk, thereby enabling targeted screening. We identified established prediction models for ACS and externally validated them in a large screening population. Methods and Results Prediction models for prevalent cases with ≥50% ACS were identified in a systematic review (975 studies reviewed and 6 prediction models identified [3 for moderate and 3 for severe ACS]) and then validated using data from 596 469 individuals who attended commercial vascular screening clinics in the United States and United Kingdom. We assessed discrimination and calibration. In the validation cohort, 11 178 (1.87%) participants had ≥50% ACS and 2033 (0.34%) had ≥70% ACS. The best model included age, sex, smoking, hypertension, hypercholesterolemia, diabetes mellitus, vascular and cerebrovascular disease, measured blood pressure, and blood lipids. The area under the receiver operating characteristic curve for this model was 0.75 (95% CI, 0.74-0.75) for ≥50% ACS and 0.78 (95% CI, 0.77-0.79) for ≥70% ACS. The prevalence of ≥50% ACS in the highest decile of risk was 6.51%, and 1.42% for ≥70% ACS. Targeted screening of the 10% highest risk identified 35% of cases with ≥50% ACS and 42% of cases with ≥70% ACS. Conclusions Individuals at high risk of significant ACS can be selected reliably using a prediction model. The best-performing prediction models identified over one third of all cases by targeted screening of individuals in the highest decile of risk only.

Keywords: atherosclerosis; carotid artery stenosis; external validation; ischemic stroke; prevention; risk prediction model; targeted screening.

Figure 1

Flowchart of literature review to identify the included studies.

Figure 2

Discriminative performance of risk prediction models. The symbols represent the AUROC curves of the included prediction models and the vertical bars represent the 95% CIs. The values of the AUROC curves and 95% CIs are provided in Table S6. The models of Jacobowitz et al24 and Qureshi et al25 were originally developed for >50% ACS and ≥60% ACS, respectively. Suri et al, 2008 used ≥50% ACS and ≥75% ACS as outcomes for the external validation.27 The AUROC curves of 2 external validations for ≥50% ACS in the models developed for ≥70% ACS by de Weerd et al23 and Yan et al26 and 2 external validations for ≥70% ACS in the models developed for ≥50% ACS by the same authors are omitted in this figure. ACS indicates asymptomatic carotid stenosis; and AUROC, area under receiver operating characteristic.

Figure 3

Clinical application of the prediction model of de Weerd et al 23 for ≥50% ACS. A, Calibration plot of external validation of the prediction model developed by de Weerd et al.23 It shows the predicted and observed prevalence of ≥50% ACS (after recalibration with adjusting the intercept). The boxes represent one decile of predicted risk, and the vertical lines represent the 95% CIs. B, Graph showing the sensitivity and specificity and corresponding observed prevalence and number needed to screen to detect 1 participant with ≥50% ACS using the prediction model developed by de Weerd et al.23 The square corresponds to targeted screening of participants in the highest decile of predicted risk. The prevalence in this decile is 6.5% with a number needed to screen of 15, and sensitivity is 34.8%. The circle corresponds to targeted screening of participants in the highest two deciles of predicted risk. The prevalence in these deciles is 4.8% with a number needed to screen of 21 and sensitivity of 55.0%. ACS indicates asymptomatic carotid stenosis; and NNS, number needed to scan.