Supporting intraoperative margin assessment using deep learning for automatic tumour segmentation in breast lumpectomy micro-PET-CT

Abstract

Complete tumour removal is vital in curative breast cancer (BCa) surgery to prevent recurrence. Recently, [¹⁸F]FDG micro-PET-CT of lumpectomy specimens has shown promise for intraoperative margin assessment (IMA). To aid interpretation, we trained a 2D Residual U-Net to delineate invasive carcinoma of no special type in micro-PET-CT lumpectomy images. We collected 53 BCa lamella images from 19 patients with true histopathology-defined tumour segmentations. Group five-fold cross-validation yielded a dice similarity coefficient of 0.71 ± 0.20 for segmentation. Afterwards, an ensemble model was generated to segment tumours and predict margin status. Comparing predicted and true histopathological margin status in a separate set of 31 micro-PET-CT lumpectomy images of 31 patients achieved an F1 score of 84%, closely matching the mean performance of seven physicians who manually interpreted the same images. This model represents an important step towards a decision-support system that enhances micro-PET-CT-based IMA in BCa, facilitating its clinical adoption.

Fig. 1. Example of a micro-PET-CT slice with the corresponding true tumour segmentation, as annotated on the whole slide image (WSI) by the pathologist, for an invasive carcinoma of no special type (NST).

On the left, the 2D micro-PET-CT image, aligned with the histopathology WSI, is visualised. The micro-PET and micro-CT images are expressed in standardised uptake values (SUVs) and Hounsfield units (HUs), respectively. In the middle, the WSI is shown. On the right, the binary tumour annotation is visualised. Malignant (NST), benign, and unknown pixels are shown in white, black, and grey, respectively.

Fig. 2. Example of the tumour segmentations predicted with intensity thresholding and Residual U-Net (ResU-Net) for an invasive carcinoma of no special type.

Predictions are shown for three different inputs: micro-CT, micro-PET, and micro-PET-CT. The micro-PET and micro-CT images are expressed in standardised uptake values (SUVs) and Hounsfield units (HUs), respectively. True positive (TP), true negative (TN), false positive (FP), and false negative (FN) pixels are shown in white, black, blue, and orange, respectively. Grey pixels indicate pixels with unknown labels.

Fig. 3. Box plot showing metric distributions assessing agreement between true and predicted tumour segmentations for test patients with invasive carcinoma of no special type.

The following metrics are shown: the dice similarity coefficient (DSC), the area under the curve of the precision-recall curve (PR-AUC), the 95th percentile of the Hausdorff distance (95HD) in mm, and the contour dice with a 1 mm tolerance (1mmCD). The metrics are shown per model input (micro-CT, micro-PET, or micro-PET-CT) and per segmentation strategy (Intensity thresholding, or Residual U-Net (ResU-Net)).

Fig. 4. Predicted tumour contours for two specimens with invasive carcinomas of no special type.

For every specimen, three orthogonal slices are visualised. The top row shows the micro-PET-CT input, and the bottom row shows the contours on top of the micro-CT. The micro-PET and micro-CT images are expressed in standardised uptake values (SUVs) and Hounsfield units (HUs), respectively. The specimen contour is shown in blue, and the contours of the tumour predictions are shown in orange and green for the Residual U-Net (ResU-Net) and intensity thresholding, respectively. a Specimen with negative histopathological margin. The ResU-Net prediction is true negative, while intensity thresholding leads to a false positive prediction. b Specimen with positive histopathological margin. Both ResU-Net and intensity thresholding predictions are true positive.

Fig. 5. Illustration of the Residual U-Net (ResU-Net) architecture used in this work.

The composition of the residual downsampling, bottleneck, and upsampling units is shown. The number of layers shown in this figure is only indicative, as a suited number of layers is defined using a grid search. The number of channels (n) is doubled for every downsampling unit and halved for every upsampling unit. We used the MONAI implementation of the architecture introduced by Kerfoot et al..

Fig. 6. Illustration of the workflow used to automatically segment the tumour and predict the margin status for a lumpectomy specimen with the deep learning (DL) model.

The DL model takes a 3D micro-PET-CT image as input, and predicts voxel-wise tumour probabilities. The predicted tumour probabilities (between 0 and 1) are here represented by a heatmap on top of the micro-CT. The predicted tumour segmentation, obtained by thresholding the predicted tumour probabilities at 0.5, is shown in orange on top of the micro-CT. A positive margin is found at the location where the tumour contour (in orange) touches the specimen contour (in blue). Every figure panel only shows one slice of the 3D image of the lumpectomy specimen.