Comprehensive comparison of stroke risk score performance: a systematic review and meta-analysis among 6 267 728 patients with atrial fibrillation

Abstract

Aims: Multiple risk scores to predict ischaemic stroke (IS) in patients with atrial fibrillation (AF) have been developed. This study aims to systematically review these scores, their validations and updates, assess their methodological quality, and calculate pooled estimates of the predictive performance.

Methods and results: We searched PubMed and Web of Science for studies developing, validating, or updating risk scores for IS in AF patients. Methodological quality was assessed using the Prediction model Risk Of Bias ASsessment Tool (PROBAST). To assess discrimination, pooled c-statistics were calculated using random-effects meta-analysis. We identified 19 scores, which were validated and updated once or more in 70 and 40 studies, respectively, including 329 validations and 76 updates-nearly all on the CHA2DS2-VASc and CHADS2. Pooled c-statistics were calculated among 6 267 728 patients and 359 373 events of IS. For the CHA2DS2-VASc and CHADS2, pooled c-statistics were 0.644 [95% confidence interval (CI) 0.635-0.653] and 0.658 (0.644-0.672), respectively. Better discriminatory abilities were found in the newer risk scores, with the modified-CHADS2 demonstrating the best discrimination [c-statistic 0.715 (0.674-0.754)]. Updates were found for the CHA2DS2-VASc and CHADS2 only, showing improved discrimination. Calibration was reasonable but available for only 17 studies. The PROBAST indicated a risk of methodological bias in all studies.

Conclusion: Nineteen risk scores and 76 updates are available to predict IS in patients with AF. The guideline-endorsed CHA2DS2-VASc shows inferior discriminative abilities compared with newer scores. Additional external validations and data on calibration are required before considering the newer scores in clinical practice.

Clinical trial registration: ID CRD4202161247 (PROSPERO).

Keywords: C-statistic; Atrial fibrillation; CHA2DS2-VASc; CHADS2; Calibration; Discrimination‌; External validation; Ischaemic stroke; Meta-analysis; Prediction model; Predictive performance; Risk score; ‌PROBAST.

Figure 1

Flowchart study selection. Two iterative searches were performed to identify (i) development studies on risk scores for IS in patients with AF (ii) corresponding validation and update studies.

Figure 2

Risk of bias (ROB) and applicability assessment using the PROBAST. (A and B) development studies (n = 19), (C and D) validation studies (n = 70), and (E and F) update studies (n = 40). Blue: high risk of bias, green: low risk of bias, pink: unclear bias due to lack of information.

Figure 3

Timeline on the development, validation, and update of risk scores. (Upper panel) The number of external validations plotted over time; (Middle panel) The number of update studies plotted over time; (lower panel) timeline of developed risk scores. Note that the number of validations slightly differs with Table 3, which gives the number of validations that were included in the random effects meta-analysis.

Figure 4

Discriminative performances over time. In this figure, the risk scores’ years of publication (x-axis) are plotted against the number of AF patients (y-axis 1, lower panel), the number of validation studies (y-axis 2, middle panel), and results of the pooled c-statistic (y-axis 3, upper panel). The horizontal lines and the marked red area in the upper panel indicate the c-statistic and 95% confidence interval of the CHADS₂ (first horizontal line) and CHA₂DS₂-VASc (second line) during the perioded these scores were endorsed by the relevant guidelines (resp 2006 to 2012 and 2012 to present). The pooled c-statistics were generated from the random-effect meta-analysis. Ten risk scores were included in the random-effects analysis and are therefore displayed in the figure. ^aWalraven and ACTS were externally validated, yet not included in the analysis as the yielded discrimination measures (c-statistic) did not include confidence intervals, which were required for the random-effects meta-analysis (more details in the Statistical analysis section). AF, atrial fibrillation.