Applications of Explainable Artificial Intelligence in Diagnosis and Surgery

Abstract

In recent years, artificial intelligence (AI) has shown great promise in medicine. However, explainability issues make AI applications in clinical usages difficult. Some research has been conducted into explainable artificial intelligence (XAI) to overcome the limitation of the black-box nature of AI methods. Compared with AI techniques such as deep learning, XAI can provide both decision-making and explanations of the model. In this review, we conducted a survey of the recent trends in medical diagnosis and surgical applications using XAI. We have searched articles published between 2019 and 2021 from PubMed, IEEE Xplore, Association for Computing Machinery, and Google Scholar. We included articles which met the selection criteria in the review and then extracted and analyzed relevant information from the studies. Additionally, we provide an experimental showcase on breast cancer diagnosis, and illustrate how XAI can be applied in medical XAI applications. Finally, we summarize the XAI methods utilized in the medical XAI applications, the challenges that the researchers have met, and discuss the future research directions. The survey result indicates that medical XAI is a promising research direction, and this study aims to serve as a reference to medical experts and AI scientists when designing medical XAI applications.

Keywords: artificial intelligence; deep learning; diagnosis; explainable artificial intelligence (XAI); machine learning; surgery.

Figure 1

The relationship between artificial intelligence, machine learning, deep learning, and explainable artificial intelligence.

Figure 2

Taxonomy of XAI methods, post hoc XAI types, and some examples.

Figure 3

The overall pipeline of a medical XAI application: the XAI methods can be intrinsic or post hoc, and they can provide decision-making and explanation to the doctors.

Figure 4

Chronic wound image and its importance map using LIME [30]: (a) original wound image; (b) importance map.

Figure 5

Visual feedback of the surgeon’s surgical task using CAM [50]. Visual feedback for the surgeon’s surgical task using CAM [50]. The red and orange subsequences in the plot show the high contribution to the surgeon’s surgical skill assessment task. In contrast, the green and blue subsequences indicate the low contribution.

Figure 6

Interpreting a prediction with the post hoc XAI method: SHAP.

Figure 7

Interpreting a prediction with the post hoc XAI method: LIME. The x-axis shows the feature effect.

Figure 8

Interpret the black-box model’s decisions with PDP for the feature “mean radius”.