Development of Electronic Health Record-Based Machine Learning Models to Predict Barrett's Esophagus and Esophageal Adenocarcinoma Risk

Abstract

Introduction: Screening for Barrett's esophagus (BE) is suggested in those with risk factors, but remains underutilized. BE/esophageal adenocarcinoma (EAC) risk prediction tools integrating multiple risk factors have been described. However, accuracy remains modest (area under the receiver-operating curve [AUROC] ≤0.7), and clinical implementation has been challenging. We aimed to develop machine learning (ML) BE/EAC risk prediction models from an electronic health record (EHR) database.

Methods: The Clinical Data Analytics Platform, a deidentified EHR database of 6 million Mayo Clinic patients, was used to predict BE and EAC risk. BE and EAC cases and controls were identified using International Classification of Diseases codes and augmented curation (natural language processing) techniques applied to clinical, endoscopy, laboratory, and pathology notes. Cases were propensity score matched to 5 independent randomly selected control groups. An ensemble transformer-based ML model architecture was used to develop predictive models.

Results: We identified 8,476 BE cases, 1,539 EAC cases, and 252,276 controls. The BE ML transformer model had an overall sensitivity, specificity, and AUROC of 76%, 76%, and 0.84, respectively. The EAC ML transformer model had an overall sensitivity, specificity, and AUROC of 84%, 70%, and 0.84, respectively. Predictors of BE and EAC included conventional risk factors and additional novel factors, such as coronary artery disease, serum triglycerides, and electrolytes.

Discussion: ML models developed on an EHR database can predict incident BE and EAC risk with improved accuracy compared with conventional risk factor-based risk scores. Such a model may enable effective implementation of a minimally invasive screening technology.

Figure 1.

Process of identifying case (BE/EAC) and control cohorts from the Clinical Data Analytics Platform. BE, Barrett's esophagus; BERT, Bidirectional Encoder Representations from Transformer; EAC, esophageal adenocarcinoma.

Figure 2.

Time line of inclusion of data for patients with BE or EAC included in model development. Anchor date was the date of diagnosis for a patient with BE or EAC. Lead time refers to a period of 1 year before the anchor date. Events in the lead time period were not used to train the model. Observation time is the period 5 years preceding the lead date. All events in the observation period were used to train the model. BE, Barrett's esophagus; EAC, esophageal adenocarcinoma.

Figure 3.

Description of case and control cohort utilization in model development. (a) Ensemble model development and architecture. Five independent control cohorts were created. Five control patients were matched to each patient with BE and 10 control patients matched to each patient with EAC. Five transformer models were developed by pairing the BE and EAC case cohort with 5 independent control cohorts. These 5 transformer models were then integrated into a single ensemble model using logistic regression. (b) Schematic showing the layers of the transformer model used to build the BE and EAC machine learning predictive models. BE, Barrett's esophagus; EAC, esophageal adenocarcinoma.

Figure 4.

Distribution of CDAP data used for model development and testing. CDAP, Clinical Data Analytics Platform.

Figure 5.

Sequential identification of BE and EAC cases from the CDAP, with the application of prespecified data sufficiency, inclusion criteria, and exclusion criteria. BE, Barrett's esophagus; CDAP, Clinical Data Analytics Platform; EAC, esophageal adenocarcinoma