Do functional status and Medicare claims data improve the predictive accuracy of an electronic health record mortality index? Findings from a national Veterans Affairs cohort

Abstract

Background: Electronic health record (EHR) prediction models may be easier to use in busy clinical settings since EHR data can be auto-populated into models. This study assessed whether adding functional status and/or Medicare claims data (which are often not available in EHRs) improves the accuracy of a previously developed Veterans Affairs (VA) EHR-based mortality index.

Methods: This was a retrospective cohort study of veterans aged 75 years and older enrolled in VA primary care clinics followed from January 2014 to April 2020 (n = 62,014). We randomly split participants into development (n = 49,612) and validation (n = 12,402) cohorts. The primary outcome was all-cause mortality. We performed logistic regression with backward stepwise selection to develop a 100-predictor base model using 854 EHR candidate variables, including demographics, laboratory values, medications, healthcare utilization, diagnosis codes, and vitals. We incorporated functional measures in a base + function model by adding activities of daily living (range 0-5) and instrumental activities of daily living (range 0-7) scores. Medicare data, including healthcare utilization (e.g., emergency department visits, hospitalizations) and diagnosis codes, were incorporated in a base + Medicare model. A base + function + Medicare model included all data elements. We assessed model performance with the c-statistic, reclassification metrics, fraction of new information provided, and calibration plots.

Results: In the overall cohort, mean age was 82.6 years and 98.6% were male. At the end of follow-up, 30,263 participants (48.8%) had died. The base model c-statistic was 0.809 (95% CI 0.805-0.812) in the development cohort and 0.804 (95% CI 0.796-0.812) in the validation cohort. Validation cohort c-statistics for the base + function, base + Medicare, and base + function + Medicare models were 0.809 (95% CI 0.801-0.816), 0.811 (95% CI 0.803-0.818), and 0.814 (95% CI 0.807-0.822), respectively. Adding functional status and Medicare data resulted in similarly small improvements among other model performance measures. All models showed excellent calibration.

Conclusions: Incorporation of functional status and Medicare data into a VA EHR-based mortality index led to small but likely clinically insignificant improvements in model performance.

Keywords: Functional status; Medicare data; Mortality prediction model; Physical function.

Fig. 1

Number of predictor variables selected within each of the 100 variable models. Abbreviations: c-statistic, concordance statistic; EHR, electronic health record. * For the base model and base + function model, diagnoses codes were obtained only from Veterans Affairs electronic health record data. For the base + Medicare and base + function + Medicare models, diagnosis codes were obtained from the Veterans Affairs electronic health record and Medicare sources