Radiomics preoperative-Fistula Risk Score (RAD-FRS) for pancreatoduodenectomy: development and external validation

Abstract

Background: Accurately predicting the risk of clinically relevant postoperative pancreatic fistula after pancreatoduodenectomy before surgery may assist surgeons in making more informed treatment decisions and improved patient counselling. The aim was to evaluate the predictive accuracy of a radiomics-based preoperative-Fistula Risk Score (RAD-FRS) for clinically relevant postoperative pancreatic fistula.

Methods: Radiomic features were derived from preoperative CT scans from adult patients after pancreatoduodenectomy at a single centre in the Netherlands (Amsterdam, 2013-2018) to develop the radiomics-based preoperative-Fistula Risk Score. Extracted radiomic features were analysed with four machine learning classifiers. The model was externally validated in a single centre in Italy (Verona, 2020-2021). The radiomics-based preoperative-Fistula Risk Score was compared with the Fistula Risk Score and the updated alternative Fistula Risk Score.

Results: Overall, 359 patients underwent a pancreatoduodenectomy, of whom 89 (25 per cent) developed a clinically relevant postoperative pancreatic fistula. The radiomics-based preoperative-Fistula Risk Score model was developed using CT scans of 118 patients, of which three radiomic features were included in the random forest model, and externally validated in 57 patients. The model performed well with an area under the curve of 0.90 (95 per cent c.i. 0.71 to 0.99) and 0.81 (95 per cent c.i. 0.69 to 0.92) in the Amsterdam test set and Verona data set respectively. The radiomics-based preoperative-Fistula Risk Score performed similarly to the Fistula Risk Score (area under the curve 0.79) and updated alternative Fistula Risk Score (area under the curve 0.79).

Conclusion: The radiomics-based preoperative-Fistula Risk Score, which uses only preoperative CT features, is a new and promising radiomics-based score that has the potential to be integrated with hospital CT report systems and improve patient counselling before surgery. The model with underlying code is readily available via www.pancreascalculator.com and www.github.com/PHAIR-Consortium/POPF-predictor.

Fig. 1

The Amsterdam data set (n = 118) was split into the Amsterdam development (90 per cent, n = 106) and test set (10 per cent, n = 12). The Amsterdam development set was further split into the Amsterdam training and validation set using five-fold cross-validation. The best out of 25 models with regards to the AUROC on the Amsterdam test set was evaluated on the Verona data set (n = 57). AUROC, area under the curve—receiver operating characteristic.

Fig. 2

An example of a contrast-enhanced CT scan in the early arterial phase of the abdomen, showing the contoured pancreatic tissue (pink), pancreatic duct (light blue), splenic artery (red) and the superior mesenteric vein (dark blue) in an axial image The vertical yellow line indicates the midline of the superior mesenteric vein, with the pancreas annotated on the left side. The volume of interest consisted of the pancreatic tissue and pancreatic duct. CT, computed tomography.

Fig. 3

Flow chart of the Amsterdam data set Flow chart showing the selection process of the Amsterdam data set. A total of 100 patients were excluded in the group without CR-POPF to create more equally sized groups based on the presence or absence of CR-POPF. Further exclusion criteria were: poor image quality (n = 3) and slice thickness above 3 mm (n = 3). A total of 50 patients with CR-POPF and 68 without CR-POPF were found eligible for analysis. PD, pancreatoduodenectomy; CR-POPF, clinically relevant postoperative pancreatic fistula; CT, computed tomography.

Fig. 4

Flow chart of the Verona data set Flow chart showing the selection process of the Verona data set. A total of 60 patients were excluded in the group without CR-POPF to create more equally sized groups based on the presence or absence of CR-POPF. Further exclusion criteria were: poor image quality (n = 6) and slice thickness above 3 mm (n = 12). A total of 22 patients with CR-POPF and 35 without CR-POPF were found eligible for analysis. PD, pancreatoduodenectomy; CR-POPF, clinically relevant postoperative pancreatic fistula; CT, computed tomography.

Fig. 5

A calibration plot of the RAD-FRS in the Amsterdam test set and Verona data set The black dots represent the quintiles of the observed probabilities by quintiles of the predicted probabilities of the Amsterdam test set, while the white triangles represent the same for the Verona data set. The dashed line represents the ideal performance of the score. RAD-FRS, radiomics preoperative-Fistula Risk Score.

Fig. 6

Comparison of the ROC curves for RAD-FRS, ua-FRS and FRS in predicting CR-POPF in the Verona data set The reference line represents the performance of a random guess. ROC, receiver operating characteristics; AUC, area under the curve; RAD-FRS, radiomics preoperative-Fistula Risk Score; ua-FRS, updated alternative Fistula Risk Score; FRS, Fistula Risk Score; CR-POPF, clinically relevant postoperative pancreatic fistula.