Early prediction of acute gallstone pancreatitis severity: a novel machine learning model based on CT features and open access online prediction platform

Abstract

Background: Early diagnosis of acute gallstone pancreatitis severity (GSP) is challenging in clinical practice. We aimed to investigate the efficacy of CT features and radiomics for the early prediction of acute GSP severity.

Methods: We retrospectively recruited GSP patients who underwent CT imaging within 48 h of admission from tertiary referral centre. Radiomics and CT features were extracted from CT scans. The clinical and CT features were selected by the random forest algorithm to develop the ML GSP model for the identification of severity of GSP (mild or severe), and its predictive efficacy was compared with radiomics model. The predictive performance was assessed by the area under operating characteristic curve. Calibration curve and decision curve analysis were performed to demonstrate the classification performance and clinical efficacy. Furthermore, we built a web-based open access GSP severity calculator. The study was registered with ClinicalTrials.gov (NCT05498961).

Results: A total of 301 patients were enrolled. They were randomly assigned into the training (n = 210) and validation (n = 91) cohorts at a ratio of 7:3. The random forest algorithm identified the level of calcium ions, WBC count, urea level, combined cholecystitis, gallbladder wall thickening, gallstones, and hydrothorax as the seven predictive factors for severity of GSP. In the validation cohort, the areas under the curve for the radiomics model and ML GSP model were 0.841 (0.757-0.926) and 0.914 (0.851-0.978), respectively. The calibration plot shows that the ML GSP model has good consistency between the prediction probability and the observation probability. Decision curve analysis showed that the ML GSP model had high clinical utility.

Conclusions: We built the ML GSP model based on clinical and CT image features and distributed it as a free web-based calculator. Our results indicated that the ML GSP model is useful for predicting the severity of GSP.

Keywords: CT features; Gallstone pancreatitis; prediction; radiomics; random forest.

Figure 1.

LASSO (least absolute shrinkage and selection operator) regression was used for radiomics features selection. (A), The selection of parameter (λ) in the LASSO model is verified by 10-fold cross-validation via minimum criteria. The relationship between the MSE (misclassification error) curve and λ is plotted. The vertical line is drawn at the optimal value by using the minimum criterion and one standard error of the minimum criterion (1-SE standard). The optimal λ value is 0.464. (B), 862 selected characteristic LASSO coefficient curves. A 10-fold cross-validation is used to draw a vertical line at the selected value, where the best λ produces 29 nonzero coefficients.

Figure 2.

The flowchart of the RF algorithm calculations. With the increase in features, the prediction performance also changes. When the top-ranking 7 features are included (the dotted line), the best AUC of the model is achieved.

Figure 3.

(A) SHAP summary plot of selected variables. Each variable of each patient was coloured by a point according to an attribute value. Red represents a higher value, and blue represents a lower value; (B) shows the importance matrix plot of the ML GSP, describing the importance of each variable in predicting GSP.

Figure 4.

Figure 4. (A) and (B) ROC curves of the ML GSP model, radiomics model, MCTSI model, and BISAP model are shown in the both training and validation cohorts. (C) Calibration curves of the ML GSP model for predicting the severity of GSP between prediction and actual classification in the training cohort and validation cohort. The 45° straight line represents an ideal model perfectly calibrated with an outcome. A closer distance between two curves indicates higher accuracy. (D) Decision curve analysis for the combined ML GSP model in the training cohort and validation cohort. The y-axis shows the net benefit. The x-axis shows the threshold probability. Within reasonable threshold probabilities, combining the RF model in the training and validation cohorts achieves a higher benefit.

Figure 5.

The web prediction tool based on ML GSP model.