Risk score overestimation: the impact of individual cardiovascular risk factors and preventive therapies on the performance of the American Heart Association-American College of Cardiology-Atherosclerotic Cardiovascular Disease risk score in a modern multi-ethnic cohort

Abstract

Aims: To evaluate the 2013 American Heart Association (AHA)-American College of Cardiology (ACC)-Atherosclerotic Cardiovascular Disease (ASCVD) risk score among four different race/ethnic groups and to ascertain which factors are most associated with risk overestimation by the AHA-ACC-ASCVD score.

Methods and results: The Multi-Ethnic Study of Atherosclerosis (MESA), a prospective community-based cohort, was used to examine calibration and discrimination of the AHA-ACC-ASCVD risk score in 6441 White, Black, Chinese, and Hispanic Americans (aged 45-79 years and free of known ASCVD at baseline). Using univariable and multivariable absolute risk regression, we modelled the impact of individual risk factors on the discordance between observed and predicted 10-year ASCVD risk. Overestimation was observed in all race/ethnic groups in MESA and was highest among Chinese (252% for women and 314% for men) and lowest in White women (72%) and Hispanic men (67%). Higher age, Chinese race/ethnicity (when compared with White), systolic blood pressure (treated and untreated), diabetes, alcohol use, exercise, lipid-lowering medication, and aspirin use were all associated with more risk overestimation, whereas family history was associated with less risk overestimation in a multivariable model (all P < 0.05).

Conclusion: The AHA-ACC-ASCVD risk score overestimates ASCVD risk among men, women, and all four race/ethnic groups evaluated in a modern American primary prevention cohort. Clinicians treating patients similar to those in MESA, particularly older individuals and those with factors associated with more risk overestimation, may consider interpreting absolute ASCVD risk estimates with caution.

Keywords: Cardiovascular risk prediction; Prevention; Risk factors.

Figure 1

Observed atherosclerotic cardiovascular disease percentage and American Heart Association-American College of Cardiology-Atherosclerotic Cardiovascular Disease risk score predicted percentage. *P-value of <0.05 for proportion test comparing American Heart Association-American College of Cardiology-Atherosclerotic Cardiovascular Disease predicted risk (total height of each bar) with the observed risk in Multi-Ethnic Study of Atherosclerosis (grey portion of bars). For example, the American Heart Association-American College of Cardiology-Atherosclerotic Cardiovascular Disease predicted risk for men was 13.9%, whereas the observed rate was 7.2%, and a two-tailed test of proportions showed these rates were significantly different at the 95% level. Reported P-value compares the overestimation (i.e. the black bars) for each risk factor. For example, the American Heart Association-American College of Cardiology-Atherosclerotic Cardiovascular Disease predicted risk (total height of each bar) was higher than the observed risk (grey portion of the bar) for females and for males, and the overestimation for males was significantly greater (P < 0.001) than the overestimation for females. P-values come from an absolute risk regression model, which is a linear regression model at the individual level, predicting a dichotomous outcome (atherosclerotic cardiovascular disease event yes/no within 10 years). It was fit as a generalized linear model with the American Heart Association-American College of Cardiology-Atherosclerotic Cardiovascular Disease risk score entered as an offset (fixing the coefficient to 1.0), with an identity link function and using robust standard errors. To this model, we added each risk factor separately. P-value for race/ethnicity categories shows joint F-test of significance.

Figure 2

Observed atherosclerotic cardiovascular disease percentage and American Heart Association-American College of Cardiology-Atherosclerotic Cardiovascular Disease risk score predicted percentage among factors not included in the risk score. *P-value of <0.05 for proportion test comparing American Heart Association-American College of Cardiology-Atherosclerotic Cardiovascular Disease predicted risk (total height of each bar) with the observed risk in Multi-Ethnic Study of Atherosclerosis (grey portion of bars). Reported P-value compares the overestimation (i.e. the black bars) for each risk factor. P-values come from an absolute risk regression model, which is a linear regression model at the individual level, predicting a dichotomous outcome (atherosclerotic cardiovascular disease event yes/no within 10 years). It was fit as a generalized linear model with the American Heart Association-American College of Cardiology-Atherosclerotic Cardiovascular Disease risk score entered as an offset (fixing the coefficient to 1.0), with an identity link function and using robust standard errors. To this model, we added each risk factor separately. P-value for education categories variables shows joint F-test of significance.

Figure 3

Observed atherosclerotic cardiovascular disease percentage and American Heart Association-American College of Cardiology-Atherosclerotic Cardiovascular Disease risk score predicted percentage by time-updated treatments. *P-value of <0.05 for proportion test comparing American Heart Association-American College of Cardiology-Atherosclerotic Cardiovascular Disease predicted risk (total height of each bar) with the observed risk in Multi-Ethnic Study of Atherosclerosis (grey portion of bars). Reported P-value compares the overestimation (i.e. the black bars) for each risk factor. P-values come from an absolute risk regression model, which is a linear regression model at the individual level, predicting a dichotomous outcome (atherosclerotic cardiovascular disease event yes/no within 10 years). It was fit as a generalized linear model with the American Heart Association-American College of Cardiology-Atherosclerotic Cardiovascular Disease risk score entered as an offset (fixing the coefficient to 1.0), with an identity link function and using robust standard errors. To this model, we added each risk factor separately.