Validation and Recalibration of Two Multivariable Prognostic Models for Survival and Independence in Acute Stroke

Abstract

Introduction: Various prognostic models have been developed for acute stroke, including one based on age and five binary variables ('six simple variables' model; SSVMod) and one based on age plus scores on the National Institutes of Health Stroke Scale (NIHSSMod). The aims of this study were to externally validate and recalibrate these models, and to compare their predictive ability in relation to both survival and independence.

Methods: Data from a large clinical trial of oxygen therapy (n = 8003) were used to determine the discrimination and calibration of the models, using C-statistics, calibration plots, and Hosmer-Lemeshow statistics. Methods of recalibration in the large and logistic recalibration were used to update the models.

Results: For discrimination, both models functioned better for survival (C-statistics between .802 and .837) than for independence (C-statistics between .725 and .735). Both models showed slight shortcomings with regard to calibration, over-predicting survival and under-predicting independence; the NIHSSMod performed slightly better than the SSVMod. For the most part, there were only minor differences between ischaemic and haemorrhagic strokes. Logistic recalibration successfully updated the models for a clinical trial population.

Conclusions: Both prognostic models performed well overall in a clinical trial population. The choice between them is probably better based on clinical and practical considerations than on statistical considerations.

Fig 1. Receiver operating characteristics (ROC) curves for survival and independence, for the total sample.

Optimally, the curve should lie towards the upper left corner of the plot. Survival: assessed at 30 days for the SSVMod and at 100 days for the NIHSSMod. Independence: assessed at 6 months for the SSVMod and at 3 months for the NIHSSMod.

Fig 2. Calibration plots for survival and independence, for the total sample and based on deciles of patient numbers.

Survival: assessed at 30 days for the SSVMod and at 100 days for the NIHSSMod. Independence: assessed at 6 months for the SSVMod and at 3 months for the NIHSSMod. For illustrative clarity, the origins for the axes vary between plots.