Automated decision making in Barrett's oesophagus: development and deployment of a natural language processing tool

Abstract

Manual decisions regarding the timing of surveillance endoscopy for premalignant Barrett's oesophagus (BO) is error-prone. This leads to inefficient resource usage and safety risks. To automate decision-making, we fine-tuned Bidirectional Encoder Representations from Transformers (BERT) models to categorize BO length (EndoBERT) and worst histopathological grade (PathBERT) on 4,831 endoscopy and 4,581 pathology reports from Guy's and St Thomas' Hospital (GSTT). The accuracies for EndoBERT test sets from GSTT, King's College Hospital (KCH), and Sandwell and West Birmingham Hospitals (SWB) were 0.95, 0.86, and 0.99, respectively. Average accuracies for PathBERT were 0.93, 0.91, and 0.92, respectively. A retrospective analysis of 1640 GSTT reports revealed a 27% discrepancy between endoscopists' decisions and model recommendations. This study underscores the development and deployment of NLP-based software in BO surveillance, demonstrating high performance at multiple sites. The analysis emphasizes the potential efficiency of automation in enhancing precision and guideline adherence in clinical decision-making.

Fig. 1. Cohort selection process for both model training and test sets.

This figure presents a consort diagram detailing the process used for selecting patient cohorts for the training set and illustrating the distribution of test samples across different trusts: GSTT, KCH and SWB.

Fig. 2. EndominerAI end-to-end data pipeline.

The diagram presents the comprehensive data pipeline of EndominerAI, showcasing the sequence from data preprocessing to follow-up prediction. After data preprocessing, the data is submitted to EndoBERT (for endoscopic data) or PathBERT (for histopathological data), the output of these models (diamonds) is then passed in to a rule-based algorithm to determine follow up timing.

Fig. 3. EndominerAI architecture in hospital environment.

This figure displays the operational framework of EndominerAI in a hospital setting. It shows how data is extracted from hospital databases, processed and returned to the user.