Development and Validation of a Machine Learning-Based Prediction Model for Detection of Biliary Atresia

Abstract

Background and aims: Biliary atresia is a rare and devastating bile duct disease that occurs during the neonatal period. Timely identification and prompt surgical intervention is critical for improving the outcome. The aim of the study was to develop a new machine learning-based prediction model for the detection of biliary atresia.

Methods: Neonates aged <100 days with cholestasis at least once were retrospectively screened in 2 tertiary referral hospitals between 2015 and 2020. Simple demographic data, routine laboratory indices, and imaging findings of ultrasonography and hepatobiliary scintigraphy were used as features in the multivariate analysis. The extreme gradient boosting (XGBoost) framework was used to develop prediction models according to the diagnostic steps.

Results: Among 1605 enrolled neonates with all-cause cholestasis, 145 (9%) were included as having biliary atresia. Direct bilirubin, gamma-glutamyl transpeptidase, abdominal sonography, and hepatobiliary scan were the most impactful features in prediction models. The Step II XGBoost model, consisting of nonimaging inputs, showed excellent discriminatory performance (area under the curve = 0.97). The Step III and IV XGBoost models showed near-perfect performances (area under the curve = 0.998 and 0.999, respectively). In external validation (n = 912 with 118 [12.9%] biliary atresia), XGBoost-based prediction models consistently showed acceptable performances. Utilizing shapley additive explanation values also provided visualized insight and explanation of the contribution of features in detecting biliary atresia. The models were integrated into a web-based diagnostic tool for case-level application.

Conclusion: We introduced a new machine learning-based prediction model for detecting biliary atresia in the largest cohorts of neonatal cholestasis.

Keywords: Biliary Atresia; Machine Learning; Neonatal Cholestasis; Prediction; XGBoost.

Figure 1

Conceptual diagnostic steps to biliary atresia among all-cause cholestatic neonates. Cholestatic neonates are, in general, initially examined by a primary health care provider. From this perspective, diagnosis of biliary atresia among cholestatic neonates comprises 3 or 4 conceptual steps. After the first diagnostic step of detecting cholestasis, initial differential process (Step II) begins by conducting nonimaging tests including GGT and prothrombin time international normalized ratio. Then, imaging tests such as abdominal sonography and HBS could be added based on the clinical judgment (Steps III and IV, respectively). Finally, clinicians decide whether the patients need invasive confirmative tests such as biopsy and intraoperative cholangiography for the confirmation of biliary atresia. During these diagnostic steps, missing values and drop-outs of nonbiliary atresia controls occur by clinical judgments. GGT, gamma-glutamyl transpeptidase; PT INR, prothrombin time international normalized ratio; XGBoost, extreme gradient boosting.

Figure 2

Flow chart of study population. DB, direct bilirubin; XGBoost, extreme gradient boosting.

Figure 3

Contributions of features on SHAP summary plots of prediction models. (A) SHAP summary plots. The SHAP value for each patient of the development dataset by input features on the SHAP summary plots. The features on the y-axis are ranked from most important to least important with their mean absolute SHAP value. The x-axis represents the SHAP value contributed by each feature and patient, and a positive value on x-axis indicates a higher impact on the prediction of biliary atresia. The purple color indicates that the individual patients' feature value is high and vice versa (yellow). Gray dots indicate missing values. (B) Partial dependence plots. The plots of the top-4 features show a marginal effect shown by the SHAP values of features in predicting biliary atresia, presented by the average curve (red line for continuous features). ALB, albumin; ALP, alkaline phosphatase; ALT, alanine aminotransferase; AST, aspartate aminotransferase; Cr, creatinine; CRP, c-reactive protein; DB, direct bilirubin; GGT, gamma-glutamyl transpeptidase; Hb, hemoglobin; HBS, hepatobiliary scan; PLT, platelet; PT INR, prothrombin time; SHAP, shapley additive explanation; TB, total bilirubin; US, ultrasonography; WBC, white blood cell.

Figure 4

Performances of prediction models and publication of web-based prediction tool. (A) Discriminatory performances of prediction models in the external data. AUC; area under the curve in the receiver-operating characteristic curve analysis. Additional results are listed in Figure A3. (B) SHAP force plot of Step II prediction on all patients (n = 2517 with 263 [10.4%] biliary atresia). (C) Example of personalized and interpretable display of biliary atresia and nonbiliary atresia cases on web-based Step II prediction tool. In addition to providing a predicted probability, the tool also individual plots for user's understanding. Break-down plot and SHAP plot for individual patients illustrate visualized patterns of a feature's contribution. AUC, area under the curve; ALB, albumin; ALP, alkaline phosphatase; ALT, alanine aminotransferase; AST, aspartate aminotransferase; Cr, creatinine; CRP, c-reactive protein; DB, direct bilirubin; FN, false negative; FP, false positive; GGT, gamma-glutamyl transpeptidase; Hb, hemoglobin; NPV, negative predictive value; PLT, platelet; PPV, positive predictive value; PT INR, prothrombin time; S.E., standard error; SHAP, shapley additive explanation; TB, total bilirubin; TN, true negative; TP, true positive; WBC, white blood cell.