Explaining the black-box smoothly-A counterfactual approach

Abstract

We propose a BlackBox Counterfactual Explainer, designed to explain image classification models for medical applications. Classical approaches (e.g., , saliency maps) that assess feature importance do not explain how imaging features in important anatomical regions are relevant to the classification decision. Such reasoning is crucial for transparent decision-making in healthcare applications. Our framework explains the decision for a target class by gradually exaggerating the semantic effect of the class in a query image. We adopted a Generative Adversarial Network (GAN) to generate a progressive set of perturbations to a query image, such that the classification decision changes from its original class to its negation. Our proposed loss function preserves essential details (e.g., support devices) in the generated images. We used counterfactual explanations from our framework to audit a classifier trained on a chest X-ray dataset with multiple labels. Clinical evaluation of model explanations is a challenging task. We proposed clinically-relevant quantitative metrics such as cardiothoracic ratio and the score of a healthy costophrenic recess to evaluate our explanations. We used these metrics to quantify the counterfactual changes between the populations with negative and positive decisions for a diagnosis by the given classifier. We conducted a human-grounded experiment with diagnostic radiology residents to compare different styles of explanations (no explanation, saliency map, cycleGAN explanation, and our counterfactual explanation) by evaluating different aspects of explanations: (1) understandability, (2) classifier's decision justification, (3) visual quality, (d) identity preservation, and (5) overall helpfulness of an explanation to the users. Our results show that our counterfactual explanation was the only explanation method that significantly improved the users' understanding of the classifier's decision compared to the no-explanation baseline. Our metrics established a benchmark for evaluating model explanation methods in medical images. Our explanations revealed that the classifier relied on clinically relevant radiographic features for its diagnostic decisions, thus making its decision-making process more transparent to the end-user.

Keywords: Chest X-ray diagnosis; Counterfactual reasoning; Explainable AI; Interpretable machine learning.

Fig. 1.

Counterfactual explanation shows “where” + “what” minimum change must be made to the input to flip the classification decision. For Pleural Effusion, we can observe vanishing of the meniscus (red) in counterfactual image as compared to the query image.

Fig. 2.

Explanation function ${𝓘}_f(\mathbf{x},\mathbf{c})$ for classifier f. Given an input image x, we generates a perturbation of the input, x_c as explanation, such that the posterior probability, f , changes from its original value, f (x), to a desired value c while satisfying the three consistency constraints.

Fig. 3.

(a) A domain-aware self-reconstruction loss with pre-trained semantic segmentation S (x) and object detection O(x) networks. (b) The self and cyclic reconstruction should retain maximum information from x.

Fig. 4.

Qualitative comparison of the counterfactual explanations generated for two classification tasks, cardiomegaly (first row) and pleural effusion (PE) (last row). The bottom labels are the classifier’s predictions for the specific task. For the input image in first column, our model generates a series of images x_c as explanations by gradually changing c in range [0, 1]. The last column presented a pixel-wise difference map between the explanations at the two extreme ends i.e., with condition c = 0 (negative decision) and with condition c = 1 (positive decision). The heatmap highlights the regions that changed the most during the transformation. For cardiomegaly, we show the heart border in yellow. For PE, we showed the results of an object detector as a bounding-box (BB) over the healthy (blue) and blunt (red) CP recess regions. The number on the top-right of the blue-BB is the Score for detecting a healthy CP recess (SCP). The number on red-BB is 1-SCP.

Fig. 5.

The plot of condition, c (desired prediction), against actual response of the classifier on generated explanations, f (x_c). Each line represents a set of input images with prediction f (x) in a given range. Plots for xGEM and cycleGAN are shown in SM-Fig. 18.

Fig. 6.

Fidelity of generated images with respect to preserving FO.

Fig. 7.

Quantitative comparison of our method against gradient-based methods. Mean area under the deletion curve (AUDC), plotted as a function of the fraction of removed pixels. A low AUDC shows a sharp drop in prediction accuracy as more pixels are deleted.

Fig. 8.

Comparison of our method against different gradient-based methods. A: Input image; B: Saliency maps from existing works; C: Our simulation of saliency map as difference map between the normal and abnormal explanation images. More examples are shown in SM-Fig. 21.

Fig. 9.

Box plots to show distributions of pairwise differences in clinical metrics, CTR for cardiomegaly and the Score of normal CP recess (SCP) for pleural effusion, before (real) and after (counterfactual) our generative counterfactual creation process. The mean value corresponds to the average causal effect of the clinical-metric on the target task. The low p-values for the dependent t-test statistics confirm the statistically significant difference in the distributions of metrics for real and counterfactual images. The mean and standard deviation for the statistic tests are summarized in SM-Table 8.

Fig. 10.

Comparing the evaluation metrics of understandability, classifier’s decision justification, visual quality, and identity preservation across the different explanation conditions.