Interobserver ground-truth variability limits performance of automated glioblastoma segmentation on [18F]FET PET

Abstract

Background: Positron emission tomography (PET) with a [¹⁸F]fluoroethyl)-L-tyrosine ([¹⁸F]FET) tracer is of growing importance in the management of glioblastoma for the estimation of tumor extent and extraction of diagnostic and prognostic parameters. Robust and accurate glioblastoma segmentation methods are essential to maximize the benefits of this imaging modality. Given the importance of setting the foreground threshold during manual tumor delineation, this study investigates the added value of incorporating such prior knowledge to guide the automated segmentation and improve performance. Two segmentation networks were trained based on the nnU-Net guidelines: one with the [¹⁸F]FET PET image as sole input, and one with an additional input channel for the threshold map. For the latter, we investigate the benefit of manually obtained thresholds and explore automated prediction and generation of such maps. A fully automated pipeline was constructed by selecting the best performing threshold prediction approach and cascading this with the tumor segmentation model.

Results: The proposed two-channel network shows increased performance with guidance of threshold maps originating from the same reader whose ground-truth tumor label the prediction is compared to (DSC = 0.901). When threshold maps were generated by a different reader, performance reverted to levels comparable to the one-channel network and inter-reader variability. The proposed full pipeline achieves results on par with current state of the art (DSC = 0.807).

Conclusions: Incorporating a threshold map can significantly improve tumor segmentation performance when it aligns well with the ground-truth label. However, the current inability to reliably reproduce these maps-both manually and automatically-or the ground-truth tumor labels, restricts the achievable accuracy for automated glioblastoma segmentation on [¹⁸F]FET PET, highlighting the need for more consistent definitions of such ground-truth delineations.

Keywords: Brain; Deep learning; Glioblastoma; Positron emission tomography; Segmentation.

Fig. 1

Overview of data partitioning and annotation strategy, including fivefold cross-validation for training, and a fully independent test set for validation of the final model

Fig. 2

Overview of the proposed approach. The full pipeline consists of initial prediction of a threshold map from the [¹⁸F]FET PET image using an automated threshold estimation network. An overview of the investigated threshold prediction networks is shown (green) in correspondence the manual segmentation workflow (orange): from the PET image, U-Net_BKG predicts the background VOI, DenseNet_TH predicts the threshold value, and U-Net_TM predicts the threshold map. The image and threshold map are subsequently fed as input channels to the segmentation network, a two-channel U-Net, for the prediction of the tumor label. A multi-slice representation of the background VOI is shown below. VOI = Volume Of Interest

Fig. 3

Bland–Altman plots illustrating the differences between ground-truth thresholds and thresholds predicted using the various approaches for automated threshold prediction. Each point corresponds to a pair of predicted and ground-truth threshold values. Inter-reader differences are shown in (d) for different pairs of readers (1–4), where each point corresponds to a pair of threshold values, both determined by a different reader. The plots display the mean difference (bias) and 95% limits of agreement. Red crosses in (a) indicate cases where the network failed to segment a background VOI, resulting in a threshold set to 0. SD = Standard Deviation

Fig. 4

Bland–Altman plots illustrating the differences between ground-truth MTV and MTV predicted using 2 C-U-Net with threshold maps generated by the various automated threshold prediction approaches. Each point corresponds to a pair of predicted and ground-truth volumes. Inter-reader differences are shown in (d) for the different pairs of readers (1–4), where each point corresponds to a pair of tumor volumes, both determined by a different reader. The plots display the mean difference (bias) and 95% limits of agreement. GT = Ground Truth; MTV = Metabolic Tumor Volume; SD = Standard Deviation

Fig. 5

Performance of the full pipeline as a function of lesion volume (a–c) and scanner (d–f). DSC = Dice Similarity Coefficient; MTV = Metabolic Tumor Volume; NSD = Normalized Surface Dice; AVE = Absolute Volume Error

Fig. 6

Example segmentations of representative subject. Threshold map from reader A, from reader B and automatically generated from U-Net_TM are shown in the first column. Tumor label predictions from 2 C-U-Net using these threshold maps are shown in the second column and compared to ground-truth labels of both readers. Overlap between labels of both readers are visualized with corresponding metrics. AVE = Absolute Volume Error; DSC = Dice Similarity Coefficient; NSD = Normalized Surface Dice