Factors Predicting Surgical Effort Using Explainable Artificial Intelligence in Advanced Stage Epithelial Ovarian Cancer

Abstract

(1) Background: Surgical cytoreduction for epithelial ovarian cancer (EOC) is a complex procedure. Encompassed within the performance skills to achieve surgical precision, intra-operative surgical decision-making remains a core feature. The use of eXplainable Artificial Intelligence (XAI) could potentially interpret the influence of human factors on the surgical effort for the cytoreductive outcome in question; (2) Methods: The retrospective cohort study evaluated 560 consecutive EOC patients who underwent cytoreductive surgery between January 2014 and December 2019 in a single public institution. The eXtreme Gradient Boosting (XGBoost) and Deep Neural Network (DNN) algorithms were employed to develop the predictive model, including patient- and operation-specific features, and novel features reflecting human factors in surgical heuristics. The precision, recall, F1 score, and area under curve (AUC) were compared between both training algorithms. The SHapley Additive exPlanations (SHAP) framework was used to provide global and local explainability for the predictive model; (3) Results: A surgical complexity score (SCS) cut-off value of five was calculated using a Receiver Operator Characteristic (ROC) curve, above which the probability of incomplete cytoreduction was more likely (area under the curve [AUC] = 0.644; 95% confidence interval [CI] = 0.598−0.69; sensitivity and specificity 34.1%, 86.5%, respectively; p = 0.000). The XGBoost outperformed the DNN assessment for the prediction of the above threshold surgical effort outcome (AUC = 0.77; 95% [CI] 0.69−0.85; p < 0.05 vs. AUC 0.739; 95% [CI] 0.655−0.823; p < 0.95). We identified “turning points” that demonstrated a clear preference towards above the given cut-off level of surgical effort; in consultant surgeons with <12 years of experience, age <53 years old, who, when attempting primary cytoreductive surgery, recorded the presence of ascites, an Intraoperative Mapping of Ovarian Cancer score >4, and a Peritoneal Carcinomatosis Index >7, in a surgical environment with the optimization of infrastructural support. (4) Conclusions: Using XAI, we explain how intra-operative decisions may consider human factors during EOC cytoreduction alongside factual knowledge, to maximize the magnitude of the selected trade-off in effort. XAI techniques are critical for a better understanding of Artificial Intelligence frameworks, and to enhance their incorporation in medical applications.

Keywords: Explainable Artificial Intelligence; complete cytoreduction; epithelial ovarian cancer; human factors; surgical complexity score.

Figure 3

Examples of SHAP Value Dependence plots for top global explainability features showing the impact of each feature value on the prediction: (a) PCI, (b) Intra-Operative Mapping, (c) Pre-Surgery CA125, (d) Size of Largest Tumor, (e) Year of Surgery. PCI, Peritoneal Carcinomatosis Index.

Figure 4

Examples of SHAP Value Dependence plots for global explainability features reflecting human factors showing the impact of each feature value on the prediction: (a) Consultant Age, (b) Years of Experience, (c) Volume Case within Cohort, (d) Site of Consultant Training.

Figure 5

Examples of SHAP Value Interaction plots related to human factors: (a) Consultant age and timing of surgery, (b) Consultant age and stage, (c) PCI and site of consultant training, (d) Patient age and volume case within cohort. PCI, Peritoneal Carcinomatosis Index.

Figure 6

Examples of SHAP Value Interaction plots related to human factors: (a) Years of consultant experience and timing of surgery, (b) Years of consultant experience and stage, (c) Size of largest tumor and EBL, (d) Intra-Operative Mapping Score and ascites. EBL, Estimated Blood Loss.

Figure 1

Receiver Operator Characteristic (ROC) curve for Surgical Complexity Score (SCS) to detect the cut-off value that predicts incomplete cytoreduction. A cut-off value of 4.5 was calculated, above which R0 resection is not achievable with a specificity of 86.5% (AUC = 0.644 CI = 0.598–0.69).

Figure 2

(a) Feature importance bar plot description of SHAP values (left) and (b) summary plot showing a set of beeswarm plots of feature distribution for global explainability of threshold SCS prediction (right). Dots correspond to the individual EOC patients. SCS, surgical complexity score; EOC, epithelial ovarian cancer.

Figure 7

Examples of Decision plots based on feature integration by the XAI model into a single risk for prediction of surgical effort. For the probability of SCS > 4, blue features have values that increased the probability, while red features decreased the probability. The combination of impacts of all features is the predicted prediction risk for above threshold surgical effort. (a,b) The odds for SCS > 4 range between 1.40- and 1.80-fold higher than normal. (c,d) the odds for SCS < 5—which means that inomplete cytoreduction is most likely to happen—range between 1.53- and 1.8-fold higher than normal. Each feature impact value represents the change in risk when that feature’s value is known versus unknown. The examples clearly demonstrate the complex interactions between patients, surgeons, and ovarian cancer-specific features. SCS, Surgical Complexity Score.

Figure 8

Schematic representation of our concept, exploring features affecting surgical effort at cytoreductive surgery for EOC. Explainable AI is employed to interpret the fine balance between factual knowledge and surgical heuristics.