Tuning and external validation of an adult congenital heart disease risk prediction model

Abstract

Aims: Adequate risk prediction can optimize the clinical management in adult congenital heart disease (ACHD). We aimed to update and subsequently validate a previously developed ACHD risk prediction model.

Methods and results: A prediction model was developed in a prospective cohort study including 602 moderately or severely complex ACHD patients, enrolled as outpatients at a tertiary centre in the Netherlands (2011-2013). Multivariable Cox regression was used to develop a model for predicting the 1-year risks of death, heart failure (HF), or arrhythmia (primary endpoint). The Boston ACHD Biobank study, a prospectively enrolled cohort (n = 749) of outpatients who visited a referral centre in Boston (2012-2017), was used for external validation. The primary endpoint occurred in 153 (26%) and 191 (28%) patients in the derivation and validation cohorts over median follow-up of 5.6 and 2.3 years, respectively. The final model included 5 out of 14 pre-specified predictors with the following hazard ratios; New York Heart Association class ≥II: 1.92 [95% confidence interval (CI) 1.28-2.90], cardiac medication 2.52 (95% CI 1.72-3.69), ≥1 reintervention after initial repair: 1.56 (95% CI 1.09-2.22), body mass index: 1.04 (95% CI 1.01-1.07), log2 N-terminal pro B-type natriuretic peptide (pmol/L): 1.48 (95% CI 1.32-1.65). At external validation, the model showed good discrimination (C-statistic 0.79, 95% CI 0.74-0.83) and excellent calibration (calibration-in-the-large = -0.002; calibration slope = 0.99).

Conclusion: These data support the validity and applicability of a parsimonious ACHD risk model based on five readily available clinical variables to accurately predict the 1-year risk of death, HF, or arrhythmia. This risk tool may help guide appropriate care for moderately or severely complex ACHD.

Keywords: Adult congenital heart disease; Biobank; Cohort study; External validation; NT-proBNP; Prediction model; Prognosis.

Figure 1

Schematic overview of the development and validation process of the ACHD risk prediction model. The ACHD risk prediction model provides estimates of the 1-year risk of death, heart failure, or arrhythmia for moderately and severely complex ACHD patients based on five clinical variables. Online web application of the prediction model is accessible at https://achdwebcalculator-updated.shinyapps.io/achdshinyweb/, or scan the QR code. ACHD, adult congenital heart disease.

Figure 2

Calibration plot of the external validation of the prediction model by the plotted probabilities of the 1-year risk of death, heart failure, or arrhythmia. Diagonal line shows ideal calibration where predicted probability is equal to the observed probability.

Figure 3

Density plot showing the distribution of the 1-year predicted probabilities for the composite outcome of death, heart failure, or arrhythmia in the development cohort and validation cohort. Mean and median (25th–75th percentile) predicted probability for each cohort is given. The predicted probabilities of adverse outcome are higher in the validation cohort, which aligns with the higher observed probability.

Figure 4

Cumulative endpoint-free survival in 1289 adults with congenital heart disease according to sextile of the ACHD prediction model 1-year predicted risk of the composite endpoint. The endpoint was a composite of all-cause mortality, heart failure (associated with hospitalization or initiation/intensification of heart failure medication) or arrhythmia (symptomatic and recorded, or associated with treatment). ACHD, adult congenital heart disease.