An explainable-by-design end-to-end AI framework based on prototypical part learning for lesion detection and classification in Digital Breast Tomosynthesis images

Abstract

Background and objective: Breast cancer is the most common cancer among women worldwide, making early detection through breast screening crucial for improving patient outcomes. Digital Breast Tomosynthesis (DBT) is an advanced radiographic technique that enhances clarity over traditional mammography by compiling multiple X-ray images into a 3D reconstruction, thereby improving cancer detection rates. However, the large data volume of DBT poses a challenge for timely analysis. This study aims to introduce a transparent AI system that not only provides a prediction but also an explanation of that prediction, expediting the analysis of DBT scans while ensuring interpretability.

Methods: The study employs a two-stage deep learning process. The first stage uses state-of-the-art Neural Network (NN) models, specifically YOLOv5 and YOLOv8, to detect lesions within the scans. An ensemble method is also explored to enhance detection capabilities. The second stage involves classifying the identified lesions using ProtoPNet, an inherently transparent NN that leverages prototypical part learning to distinguish between benign and cancerous lesions. The system facilitates clear interpretability in decision-making, which is crucial for medical diagnostics.

Results: The performance of the AI system demonstrates competitive metric results for both detection and classification tasks (a recall of 0.76 and an accuracy of 0.70, respectively). The evaluation metrics, together with the validation by expert radiologists through clinical feedback, highlight the potential of the system for future clinical relevance. Despite challenges such as dataset limitations and the need for more accurate ground truth annotations, which limit the final values of the metrics, the approach shows significant advancement in applying AI to DBT scans.

Conclusions: This study contributes to the growing field of AI in breast cancer screening by emphasizing the need for systems that are not only accurate but also transparent and interpretable. The proposed AI system marks a significant step forward in the timely and accurate analysis of DBT scans, with potential implications for improving early breast cancer detection and patient outcomes.

Keywords: Ante-hoc explainability; DBT; Deep learning; Lesion classification; Lesion detection; ProtoPNet; XAI.

Graphical abstract

Fig. 1

Post-hoc vs. ante-hoc explainability: post-hoc methods (such as Lime, SHAP, and Grad-cam) try to explain an already existing black-box model; on the other hand, ante-hoc explainability tries to build inherently-explainable models.

Fig. 2

Visual representation of our proposed two-stage framework. Given a slice from a DBT scan, the detection module identifies the lesions. Subsequently, these lesions are cropped and fed into ProtoPNet, which performs explainable classification by indicating why a lesion is classified as benign or cancerous, based on its resemblance to learned prototypical patterns.

Fig. 3

Example of some of the slices used as inter-slice data augmentation on a lesion. For visibility purposes, we show its effect on lesion crops. Different slices of the same lesion are used to enhance the variety of the dataset.

Fig. 4

Example of some offline standard data augmentation transformations applied to a lesion crop for the classification step. Represented are the original image (a), applied shift (b), applied rotation (c), applied skew (d), applied shear (e), applied flip up-down (f), applied shift left-right (g), applied histogram equalization (h), applied brightness modification (i), and applied contrast modification (j).

Fig. 5

Example of the application of CLAHE to a DBT slice, a preprocessing step applied to images for the classification module. In (a) is the original image, and in (b) is the image with CLAHE.

Fig. 6

Representation of ProtoPNet Architecture. Initially, convolutional layers extract semantically meaningful features from the input image. Subsequently, a prototype layer evaluates these features against class-specific prototypes, i.e., distinctive patterns learned during training via clustering algorithms. Each comparison yields a similarity score. Finally, a fully-connected layer integrates these scores to predict the class with the highest cumulative similarity, thus determining the image's classification.

Fig. 7

This figure illustrates how ProtoPNet successfully identifies a lesion as malignant. It does so by comparing patches of the image with the learned prototypes. Displayed within the figure are the top three prototypes that most closely match the lesion, arranged in order of their similarity, from highest to lowest.

Fig. 8

On the left, the image displays the ground truth bounding boxes, while on the right, the bounding boxes predicted by YOLOv5 are shown. This example highlights a lesion (the one at the bottom) that, according to the official guidelines, should not be visible in this slice, since they state that a lesion spans 25% of the volume slices in each direction from the annotated slice. Contrary to the guidelines, the lesion is visible, and YOLOv5 accurately identifies it, indicating an error in the ground truth annotation for this particular slice.

Fig. 9

On the left, the image displays the ground truth bounding boxes, while on the right, the bounding boxes predicted by YOLOv5 are shown. This example highlights a lesion (the one at the top) with low positive predictive power (PPV) that was not present in the ground truth annotations, but was identified by one of our models.

Fig. 10

Two examples of scores obtained from clinical feedback on the significance of prototypes. The training images from which the prototype (in the yellow box) was selected are shown. Below each image are the scores assigned by Radiologist 1 (R1) and Radiologist 2 (R2). (a) Example of a prototype receiving a high score. (b) Example of a prototype receiving a low score.

Fig. 11

Example of a similarity utilized by ProtoPNet during classification, which received high scores from both Radiologist 1 (R1) and Radiologist 2 (R2), indicating that it adheres to their concept of similarity.

Fig. 12

Example of a similarity utilized by ProtoPNet during classification, which received low scores from both Radiologist 1 (R1) and Radiologist 2 (R2), indicating that it does not adhere to their concept of similarity.