Skip to main page content
U.S. flag

An official website of the United States government

Dot gov

The .gov means it’s official.
Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you’re on a federal government site.

Https

The site is secure.
The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

Access keys NCBI Homepage MyNCBI Homepage Main Content Main Navigation
. 2017 Apr 24;6(4):e003670.
doi: 10.1161/JAHA.116.003670.

Use and Customization of Risk Scores for Predicting Cardiovascular Events Using Electronic Health Record Data

Julian Wolfson  1 , David M Vock  2 , Sunayan Bandyopadhyay  3 , Thomas Kottke  4   5 , Gabriela Vazquez-Benitez  4 , Paul Johnson  6 , Gediminas Adomavicius  6 , Patrick J O'Connor  7
Affiliations

Use and Customization of Risk Scores for Predicting Cardiovascular Events Using Electronic Health Record Data

Julian Wolfson et al. J Am Heart Assoc. .

Abstract

Background: Clinicians who are using the Framingham Risk Score (FRS) or the American College of Cardiology/American Heart Association Pooled Cohort Equations (PCE) to estimate risk for their patients based on electronic health data (EHD) face 4 questions. (1) Do published risk scores applied to EHD yield accurate estimates of cardiovascular risk? (2) Are FRS risk estimates, which are based on data that are up to 45 years old, valid for a contemporary patient population seeking routine care? (3) Do the PCE make the FRS obsolete? (4) Does refitting the risk score using EHD improve the accuracy of risk estimates?

Methods and results: Data were extracted from the EHD of 84 116 adults aged 40 to 79 years who received care at a large healthcare delivery and insurance organization between 2001 and 2011. We assessed calibration and discrimination for 4 risk scores: published versions of FRS and PCE and versions obtained by refitting models using a subset of the available EHD. The published FRS was well calibrated (calibration statistic K=9.1, miscalibration ranging from 0% to 17% across risk groups), but the PCE displayed modest evidence of miscalibration (calibration statistic K=43.7, miscalibration from 9% to 31%). Discrimination was similar in both models (C-index=0.740 for FRS, 0.747 for PCE). Refitting the published models using EHD did not substantially improve calibration or discrimination.

Conclusions: We conclude that published cardiovascular risk models can be successfully applied to EHD to estimate cardiovascular risk; the FRS remains valid and is not obsolete; and model refitting does not meaningfully improve the accuracy of risk estimates.

Keywords: electronic health record data; risk prediction; risk score.

PubMed Disclaimer

Figures

Figure 1
Figure 1
Distribution of follow‐up times for individuals who were censored (left panel) and experienced cardiovascular events (right panel).
Figure 2
Figure 2
Calibration curves for original and refitted versions of Framingham Risk Score (FRS). Dashed and dotted lines (for original and refitted models, respectively) represent smooth curves approximating the relationship between the average predicted 5‐year risk and observed risk. The solid line, with slope 1, represents a hypothetical model with perfect calibration. CV indicates cardiovascular.
Figure 3
Figure 3
Calibration curves for original and refitted versions of Pooled Cohort Equations (PCE). Dashed and dotted lines (for original and refitted models, respectively) represent smooth curves approximating the relationship between the average predicted 5‐year risk and observed risk. The solid line, with slope 1, represents a hypothetical model with perfect calibration. CV indicates cardiovascular.
Figure 4
Figure 4
Predicted 5‐year cardiovascular (CV) risk and observed 5‐year CV event rate, by age group, for original and refitted versions of Framingham Risk Score (FRS).
Figure 5
Figure 5
Predicted 5‐year cardiovascular (CV) risk and observed 5‐year CV event rate, by age group, for original and refitted versions of Pooled Cohort Equations (PCE).
See this image and copyright information in PMC

Comment in

References

    1. Stone NJ, Robinson J, Lichtenstein AH, Bairey Merz CN, Lloyd‐Jones DM, Blum CB, McBride P, Eckel RH, Schwartz JS, Goldberg AC, Shero ST, Gordon D, Smith SC, Levy D, Watson K, Wilson PWF. 2013 ACC/AHA guideline on the treatment of blood cholesterol to reduce atherosclerotic cardiovascular risk in adults: a report of the American College of Cardiology/American Heart Association Task Force on Practice Guidelines. J Am Coll Cardiol. 2014;63:2889–2934. - PubMed
    1. Goff DC, Lloyd‐Jones DM, Bennett G, Coady S, D'Agostino RB, Gibbons R, Greenland P, Lackland DT, Levy D, O'Donnell CJ, Robinson J, Schwartz JS, Shero ST, Smith SC, Sorlie P, Stone NJ, Wilson PWF. 2013 ACC/AHA guideline on the assessment of cardiovascular risk: a report of the American College of Cardiology/American Heart Association Task Force on Practice Guidelines. Circulation. 2014;129:S49–S73. - PubMed
    1. D'Agostino RB, Vasan RS, Pencina MJ, Wolf PA, Cobain M, Massaro JM, Kannel WB. General cardiovascular risk profile for use in primary care: the Framingham Heart Study. Circulation. 2008;117:743–753. - PubMed
    1. D'Agostino RB, Grundy S, Sullivan LM, Wilson P. Validation of the Framingham coronary heart disease prediction scores: results of a multiple ethnic groups investigation. JAMA. 2001;286:180–187. - PubMed
    1. Ford ES, Ajani UA, Croft JB, Critchley JA, Labarthe DR, Kottke TE, Giles WH, Capewell S. Explaining the decrease in U.S. deaths from coronary disease, 1980–2000. N Engl J Med. 2007;356:2388–2398. - PubMed

MeSH terms

LinkOut - more resources