A multi-stage machine learning model for diagnosis of esophageal manometry

Abstract

High-resolution manometry (HRM) is the primary procedure used to diagnose esophageal motility disorders. Its manual interpretation and classification, including evaluation of swallow-level outcomes and then derivation of a study-level diagnosis based on Chicago Classification (CC), may be limited by inter-rater variability and inaccuracy of an individual interpreter. We hypothesized that an automatic diagnosis platform using machine learning and artificial intelligence approaches could be developed to accurately identify esophageal motility diagnoses. Further, a multi-stage modeling framework, akin to the step-wise approach of the CC, was utilized to leverage advantages of a combination of machine learning approaches including deep-learning models and feature-based models. Models were trained and tested using a dataset comprised of 1741 patients' HRM studies with CC diagnoses assigned by expert physician raters. In the swallow-level stage, three models based on convolutional neural networks (CNNs) were developed to predict swallow type and swallow pressurization (test accuracies of 0.88 and 0.93, respectively), and integrated relaxation pressure (IRP)(regression model with test error of 4.49 mmHg). At the study-level stage, model selection from families of the expert-knowledge-based rule models, xgboost models and artificial neural network(ANN) models were conducted. A simple model-agnostic strategy of model balancing motivated by Bayesian principles was utilized, which gave rise to model averaging weighted by precision scores. The averaged (blended) models and individual models were compared and evaluated, of which the best performance on test dataset is 0.81 in top-1 prediction, 0.92 in top-2 predictions. This is the first artificial-intelligence style model to automatically predict esophageal motility (CC) diagnoses from HRM studies using raw multi-swallow data and it achieved high accuracy. Thus, this proposed modeling framework could be broadly applied to assist with HRM interpretation in a clinical setting.

Keywords: Artificial intelligence; High-resolution manometry; Model averaging.

Figure 1:

Illustration of high-resolution manometry procedure and associated information flow from raw data to final diagnosis in current clinical practices (reproduced with permission from [14]).

Figure 2:

Explorer of HRM swallow showing the variation of pressure pattern by swallow type (Upper) and by swallow pressurization (Lower left). Lower right shows swallows with low integrated relaxation pressure (IRP) and high IRP, respectively. The calculation of IRP involves manual identifications of several landmarks. (Part of the figure is reproduced with permission from [14].)

Figure 3:

Illustration of the manually-designed decision rules based on Chicago Classification, referred to as the rule-based model. The P[E] denotes the probability of event, E. The dashed-line arrow represents Yes branch, whereas the solid-line arrow the No branch under each decision condition. Three key parameters, (a₁, a₂, a₃) were set as (15.0,0.2,0.5) in the selected model. Like the Chicago Classification, the output of this rule-based model is of 10 category, which is then merged to 8 categories (DES was merged into T3A and FRP was merged into IEM). Further details on labels could be found in Table. 1.

Figure 4:

Training history of the swallow-level AI models and their evaluation based on Confusion matrix on testing dataset. (a)

Figure 5:

Evaluation of study-level models selected from each family based on confusion matrix on the testing dataset

Figure 6:

Examples of misclassified studies. Two test swallows from two patients (A and B) are included. Patient A had a true label of EGJ outflow obstruction (EGJOO), while was predicted to have type III (spastic) achalasia (T3A). Patient B had a true label of EGJOO, while was predicted to have normal motility (NEM).