Machine learning models compared to existing criteria for noninvasive prediction of endoscopic retrograde cholangiopancreatography-confirmed choledocholithiasis

Abstract

Background and aims: Noninvasive predictors of choledocholithiasis have generally exhibited marginal performance characteristics. We aimed to identify noninvasive independent predictors of endoscopic retrograde cholangiopancreatography (ERCP)-confirmed choledocholithiasis and accordingly developed predictive machine learning models (MLMs).

Methods: Clinical data of consecutive patients undergoing first-ever ERCP for suspected choledocholithiasis from 2015-2019 were abstracted from a prospectively-maintained database. Multiple logistic regression was used to identify predictors of ERCP-confirmed choledocholithiasis. MLMs were then trained to predict ERCP-confirmed choledocholithiasis using pre-ERCP ultrasound (US) imaging only and separately using all available noninvasive imaging (US/CT/magnetic resonance cholangiopancreatography). The diagnostic performance of American Society for Gastrointestinal Endoscopy (ASGE) "high-likelihood" criteria was compared to MLMs.

Results: We identified 270 patients (mean age 46 years, 62.2% female, 73.7% Hispanic/Latino, 59% with noninvasive imaging positive for choledocholithiasis) with native papilla who underwent ERCP for suspected choledocholithiasis, of whom 230 (85.2%) were found to have ERCP-confirmed choledocholithiasis. Logistic regression identified choledocholithiasis on noninvasive imaging (odds ratio (OR) = 3.045, P = 0.004) and common bile duct (CBD) diameter on noninvasive imaging (OR=1.157, P = 0.011) as predictors of ERCP-confirmed choledocholithiasis. Among the various MLMs trained, the random forest-based MLM performed best; sensitivity was 61.4% and 77.3% and specificity was 100% and 75.0%, using US-only and using all available imaging, respectively. ASGE high-likelihood criteria demonstrated sensitivity of 90.9% and specificity of 25.0%; using cut-points achieving this specificity, MLMs achieved sensitivity up to 97.7%.

Conclusions: MLMs using age, sex, race, presence of diabetes, fever, body mass index (BMI), total bilirubin, maximum CBD diameter, and choledocholithiasis on pre-ERCP noninvasive imaging predict ERCP-confirmed choledocholithiasis with good sensitivity and specificity and outperform the ASGE criteria for patients with suspected choledocholithiasis.

Keywords: Bile duct disorders; Common bile duct stones; Endoscopic retrograde cholangiopancreatography (ERCP); Gallstones; Machine learning models (MLMs); Noninvasive imaging.

Fig. 1

ERCP indication and outcome flow diagram. Study flow diagram demonstrating proportion of ERCPs performed for suspected choledocholithiasis, presence or absence of choledocholithiasis on noninvasive imaging, and subsequent ERCP result (confirmed choledocholithiasis or absence thereof). Abbreviations: CBD, common bile duct; ERCP, endoscopic retrograde cholangiopancreatography; US, ultrasound.

Fig. 2

Multiple logistic regression model with demographic, biochemical, and radiological predictors of ERCP-confirmed choledocholithiasis assessed using (a) US only and (b) all available noninvasive imaging modalities. Abbreviations: CBD, common bile duct; ERCP, endoscopic retrograde cholangiopancreatography; US, ultrasound.

Fig. 3

Receiver operator characteristic curves for MLMs trained to predict the presence of ERCP-confirmed choledocholithiasis. Model fit using predictors assessed on (a) US only and (b) all available noninvasive imaging modalities. AUC could not be calculated for the ASGE high-likelihood criteria because applying the criteria to our dataset does not generate class membership properties. The performance of the ASGE criteria is expressed as a single point here. Abbreviations: ASGE, American Society for Gastroenterology; AUC, area under the receiver operator characteristic curve; ERCP, endoscopic retrograde cholangiopancreatography; GLM, generalized linear model; MLM, machine learning model; RBF, radial basis function; SVM, support vector machine; US, ultrasound.