Skip to main page content
U.S. flag

An official website of the United States government

Dot gov

The .gov means it’s official.
Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you’re on a federal government site.

Https

The site is secure.
The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

Access keys NCBI Homepage MyNCBI Homepage Main Content Main Navigation
. 2024 Jan 24;27(2):109022.
doi: 10.1016/j.isci.2024.109022. eCollection 2024 Feb 16.

Evaluation of GPT-4 for 10-year cardiovascular risk prediction: Insights from the UK Biobank and KoGES data

Changho Han  1 , Dong Won Kim  1 , Songsoo Kim  1 , Seng Chan You  1   2 , Jin Young Park  3   4   5 , SungA Bae  3   6 , Dukyong Yoon  1   2   3
Affiliations

Evaluation of GPT-4 for 10-year cardiovascular risk prediction: Insights from the UK Biobank and KoGES data

Changho Han et al. iScience. .

Abstract

Cardiovascular disease (CVD) remains a pressing global health concern. While traditional risk prediction methods such as the Framingham and American College of Cardiology/American Heart Association (ACC/AHA) risk scores have been widely used in the practice, artificial intelligence (AI), especially GPT-4, offers new opportunities. Utilizing large scale of multi-center data from 47,468 UK Biobank participants and 5,718 KoGES participants, this study quantitatively evaluated the predictive capabilities of GPT-4 in comparison with traditional models. Our results suggest that the GPT-based score showed commendably comparable performance in CVD prediction when compared to traditional models (AUROC on UKB: 0.725 for GPT-4, 0.733 for ACC/AHA, 0.728 for Framingham; KoGES: 0.664 for GPT-4, 0.674 for ACC/AHA, 0.675 for Framingham). Even with omission of certain variables, GPT-4's performance was robust, demonstrating its adaptability to data-scarce situations. In conclusion, this study emphasizes the promising role of GPT-4 in predicting CVD risks across varied ethnic datasets, pointing toward its expansive future applications in the medical practice.

Keywords: Artificial intelligence; Cardiovascular medicine; Health informatics; Health sciences; Health technology; Medicine.

PubMed Disclaimer

Conflict of interest statement

The authors declare no competing interests.

Figures

None
Graphical abstract
Figure 1
Figure 1
Example of a GPT prompt and response Tabular data extracted from UK Biobank and KoGES were organized and queried into a sentence format. The 10-year cardiovascular disease risk percentage was extracted using regular expressions from the corresponding answers. GPT: generative pretrained transformer, UK: United Kingdom, KoGES: Korean Genome and Epidemiology Study, HDL: high-density lipoprotein, LDL: low-density lipoprotein, BMI: body mass index.
Figure 2
Figure 2
Performance evaluation and comparison of the risk scoring methods in the UK Biobank and KoGES cohorts GPT-4’s performance was comparable to conventional risk prediction models. Substantial correlation was found between the GPT-based risk score, ACC/AHA risk score and Framingham risk score. (A) AUROC curves (UK Biobank). (B) AUROC curves (KoGES). (C) Scatterplot (UK Biobank, GPT-4 vs. ACC/AHA risk score). (D) Scatterplot (KoGES, GPT-4 vs. ACC/AHA risk score). UK: United Kingdom, KoGES: Korean Genome and Epidemiology Study, GPT: generative pretrained transformer, ACC/AHA: American College of Cardiology/American Heart Association, AUROC: area under the receiver operating characteristics curve.
Figure 3
Figure 3
Kaplan-Meier curves stratified by risk categories in the UK Biobank cohort All pairwise comparisons between curves with the log rank test with post-hoc Bonferroni correction were statistically significant. (A) Kaplan-Meier curve stratified by GPT-4 based risk category. (B) Kaplan-Meier curve stratified by Framingham risk score category. (C) Kaplan-Meier curve stratified by ACC/AHA risk score category. GPT: generative pretrained transformer, ACC/AHA: American College of Cardiology/American Heart Association, UK: United Kingdom.
See this image and copyright information in PMC

References

    1. Timmis A., Vardas P., Townsend N., Torbica A., Katus H., De Smedt D., Gale C.P., Maggioni A.P., Petersen S.E., Huculeci R., et al. European Society of Cardiology: cardiovascular disease statistics 2021. Eur. Heart J. 2022;43:716–799. doi: 10.1093/eurheartj/ehab892. - DOI - PubMed
    1. Piepoli M.F., Hoes A.W., Agewall S., Albus C., Brotons C., Catapano A.L., Cooney M.-T., Corrà U., Cosyns B., Deaton C., et al. 2016 European Guidelines on cardiovascular disease prevention in clinical practice. Rev. Esp. Cardiol. 2016;69:939. doi: 10.1016/j.rec.2016.09.009. - DOI - PubMed
    1. D’Agostino R.B., Vasan R.S., Pencina M.J., Wolf P.A., Cobain M., Massaro J.M., Kannel W.B. General cardiovascular risk profile for use in primary care: the Framingham Heart Study. Circulation. 2008;117:743–753. doi: 10.1161/CIRCULATIONAHA.107.699579. - DOI - PubMed
    1. Goff D.C., Jr., Lloyd-Jones D.M., Bennett G., Coady S., D’Agostino R.B., Gibbons R., Greenland P., Lackland D.T., Levy D., O’Donnell C.J., et al. 2013 ACC/AHA guideline on the assessment of cardiovascular risk: a report of the American College of Cardiology/American Heart Association Task Force on Practice Guidelines. Circulation. 2014;129:S49–S73. doi: 10.1161/01.cir.0000437741.48606.98. - DOI - PubMed
    1. Rajkomar A., Dean J., Kohane I. Machine Learning in Medicine. N. Engl. J. Med. 2019;380:1347–1358. doi: 10.1056/NEJMra1814259. - DOI - PubMed

LinkOut - more resources