Independent external validation of cardiovascular disease mortality in women utilising Framingham and SCORE risk models: a mortality follow-up study

Abstract

Background: We conducted an independent external validation of three cardiovascular risk score models (Framingham risk score model and SCORE risk charts developed for low-risk regions and high-risk regions in Europe) on a prospective cohort of 4487 Australian women with no previous history of heart disease, diabetes or stroke. External validation is an important step to evaluate the performance of risk score models using discrimination and calibration measures to ensure their applicability beyond the settings in which they were developed.

Methods: Ten year mortality follow-up of 4487 Australian adult women from the National Heart Foundation third Risk Factor Prevalence Study with no baseline history of heart disease, diabetes or stroke. The 10-year risk of cardiovascular mortality was calculated using the Framingham and SCORE models and the predictive accuracy of the three risk score models were assessed using both discrimination and calibration.

Results: The discriminative ability of the Framingham and SCORE models were good (area under the curve > 0.85). Although all models overestimated the number of cardiovascular deaths by greater than 15%, the Hosmer-Lemeshow test indicated that the Framingham and SCORE-Low models were calibrated and hence suitable for predicting the 10-year cardiovascular mortality risk in this Australian population. An assessment of the treatment thresholds for each of the three models in identifying participants recommended for treatment were found to be inadequate, with low sensitivity and high specificity resulting from the high recommended thresholds. Lower treatment thresholds of 8.7% for the Framingham model, 0.8% for the SCORE-Low model and 1.3% for the SCORE-High model were identified for each model using the Youden index, at greater than 78% sensitivity and 80% specificity.

Conclusions: Framingham risk score model and SCORE risk chart for low-risk regions are recommended for use in the Australian women population for predicting the 10-year cardiovascular mortality risk. These models demonstrate good discrimination and calibration performance. Lower treatment thresholds are proposed for better identification of individuals for treatment.

Figure 1

Comparison of predicted and observed 10-year CVD death risk by age category. (Framingham predicted risk – red line, SCORE-Low predicted risk – green line, SCORE-High predicted risk – purple line and observed risk – black dotted line).

Figure 2

Comparison of predicted and observed 10-year CVD death risk by quintiles of risk (Predicted risk – red line and observed risk – black dotted line).