Outcome Prediction Models for Endovascular Treatment of Ischemic Stroke: Systematic Review and External Validation

Abstract

Background and purpose: Prediction models for outcome of patients with acute ischemic stroke who will undergo endovascular treatment have been developed to improve patient management. The aim of the current study is to provide an overview of preintervention models for functional outcome after endovascular treatment and to validate these models with data from daily clinical practice.

Methods: We systematically searched within Medline, Embase, Cochrane, Web of Science, to include prediction models. Models identified from the search were validated in the MR CLEAN (Multicenter Randomized Clinical Trial of Endovascular Treatment for Acute Ischemic Stroke in the Netherlands) registry, which includes all patients treated with endovascular treatment within 6.5 hours after stroke onset in the Netherlands between March 2014 and November 2017. Predictive performance was evaluated according to discrimination (area under the curve) and calibration (slope and intercept of the calibration curve). Good functional outcome was defined as a score of 0-2 or 0-3 on the modified Rankin Scale depending on the model.

Results: After screening 3468 publications, 19 models were included in this validation. Variables included in the models mainly addressed clinical and imaging characteristics at baseline. In the validation cohort of 3156 patients, discriminative performance ranged from 0.61 (SPAN-100 [Stroke Prognostication Using Age and NIH Stroke Scale]) to 0.80 (MR PREDICTS). Best-calibrated models were THRIVE (The Totaled Health Risks in Vascular Events; intercept -0.06 [95% CI, -0.14 to 0.02]; slope 0.84 [95% CI, 0.75-0.95]), THRIVE-c (intercept 0.08 [95% CI, -0.02 to 0.17]; slope 0.71 [95% CI, 0.65-0.77]), Stroke Checkerboard score (intercept -0.05 [95% CI, -0.13 to 0.03]; slope 0.97 [95% CI, 0.88-1.08]), and MR PREDICTS (intercept 0.43 [95% CI, 0.33-0.52]; slope 0.93 [95% CI, 0.85-1.01]).

Conclusions: The THRIVE-c score and MR PREDICTS both showed a good combination of discrimination and calibration and were, therefore, superior in predicting functional outcome for patients with ischemic stroke after endovascular treatment within 6.5 hours. Since models used different predictors and several models had relatively good predictive performance, the decision on which model to use in practice may also depend on simplicity of the model, data availability, and the comparability of the population and setting.

Keywords: calibration; ischemic stroke; population; prognosis; publications; systematic review.

Figure 1.

Flowchart of the selected prediction models after a systematic search of the literature. mRS indicates modified Rankin Scale.

Figure 2.

Overview of the area under the curve (AUC) ranked by discriminative performance for different Rankin Scale score cut points separately. Red: poor discrimination (AUC, 0.6–0.64), orange: poor discrimination (0.65–0.69), green: acceptable discrimination (0.70–0.79), and dark green: excellent discrimination (0.80 or higher).^– Per model is described how many variables were in the final model, and whether they included comorbidities and/or radiological variables in their model. mRS indicates modified Rankin Scale; SC, Stroke Checkerboard; and TVSS, Tor Vergata Stroke Score.

Figure 3.

Predicted vs observed proportion of good functional outcome measured by the modified Rankin Scale (mRS) for included models. A, Models that predicted outcomes with logistic regression (MR PREDICTS and THRIVE-c). B, Risk scores with a calculation of points for certain risk categories with accompanying risks of good functional outcome (%) for mRS score 0–3. C, For mRS score 0–2. For risk scores, the predicted vs observed proportions of patients with a good functional outcome were analyzed since no calibration graph could be derived because the model output is not probabilistic.