Benefits of spatial uncertainty aggregation for segmentation in digital pathology

Abstract

Purpose: Uncertainty estimation has gained significant attention in recent years for its potential to enhance the performance of deep learning (DL) algorithms in medical applications and even potentially address domain shift challenges. However, it is not straightforward to incorporate uncertainty estimation with a DL system to achieve a tangible positive effect. The objective of our work is to evaluate if the proposed spatial uncertainty aggregation (SUA) framework may improve the effectiveness of uncertainty estimation in segmentation tasks. We evaluate if SUA boosts the observed correlation between the uncertainty estimates and false negative (FN) predictions. We also investigate if the observed benefits can translate to tangible improvements in segmentation performance.

Approach: Our SUA framework processes negative prediction regions from a segmentation algorithm and detects FNs based on an aggregated uncertainty score. It can be utilized with many existing uncertainty estimation methods to boost their performance. We compare the SUA framework with a baseline of processing individual pixel's uncertainty independently.

Results: The results demonstrate that SUA is able to detect FN regions. It achieved $F_{\beta =0.5}$ of 0.92 on the in-domain and 0.85 on the domain-shift test data compared with 0.81 and 0.48 achieved by the baseline uncertainty, respectively. We also demonstrate that SUA yields improved general segmentation performance compared with utilizing the baseline uncertainty.

Conclusions: We propose the SUA framework for incorporating and utilizing uncertainty estimates for FN detection in DL segmentation algorithms for histopathology. The evaluation confirms the benefits of our approach compared with assessing pixel uncertainty independently.

Keywords: computational pathology; deep learning; false negative detection; tumor metastases segmentation; uncertainty estimation.

Fig. 1

Visualization of NPRs. The leftmost image shows a part of the original WSI, and the middle one shows the segmentation heatmap produced by a DL model. In the rightmost image, NPRs are visualized in different colors. They are determined by finding adjacent pixels that are assigned a segmentation softmax score in a predetermined range (0.55 to 0.95 in this example).

Fig. 2

Outline of the SUA framework that utilizes spatially aggregated uncertainty to identify FNs.

Fig. 3

Histogram of the 90th percentile entropy of the NPRs built using varying softmax ranges. FNs are the islands that have at least 90% overlap with the ground truth tumor annotation. Camelyon validation data.

Fig. 4

Histogram of the average entropy of the NPRs built using varying softmax ranges. FNs are the islands that have at least 90% overlap with the ground truth tumor annotation. Camelyon validation data.

Fig. 5

Histogram of the baseline pixel entropy divided between TN and FN predictions. Pixels analyzed with softmax values in the ranges of 0.55 to 0.95, 0.65 to 0.95, 0.75 to 0.95, and 0.85 to 0.95. Camelyon validation data.

Fig. 6

FNCR achieved by SUA and the baseline on the Camelyon and Sentinel datasets with 1000 bootstrap iterations. The red horizontal line in each box indicates the median value. The results are reported per considered softmax ranges, i.e., 0.55 to 0.95, 0.65 to 0.95, 0.75 to 0.95, and 0.85 to 0.95.

Fig. 7

Example of a refined WSI from the Camelyon data via the SUA framework based on NPRs with the 0.55 to 0.95 softmax range. “Tumor mask” is the ground truth tumor annotation.