A convolutional neural network with self-attention for fully automated metabolic tumor volume delineation of head and neck cancer in [ 18 F]FDG PET/CT

Abstract

Purpose: PET-derived metabolic tumor volume (MTV) and total lesion glycolysis of the primary tumor are known to be prognostic of clinical outcome in head and neck cancer (HNC). Including evaluation of lymph node metastases can further increase the prognostic value of PET but accurate manual delineation and classification of all lesions is time-consuming and prone to interobserver variability. Our goal, therefore, was development and evaluation of an automated tool for MTV delineation/classification of primary tumor and lymph node metastases in PET/CT investigations of HNC patients.

Methods: Automated lesion delineation was performed with a residual 3D U-Net convolutional neural network (CNN) incorporating a multi-head self-attention block. 698 ${[}^{18}$F]FDG PET/CT scans from 3 different sites and 5 public databases were used for network training and testing. An external dataset of 181 ${[}^{18}$F]FDG PET/CT scans from 2 additional sites was employed to assess the generalizability of the network. In these data, primary tumor and lymph node (LN) metastases were interactively delineated and labeled by two experienced physicians. Performance of the trained network models was assessed by 5-fold cross-validation in the main dataset and by pooling results from the 5 developed models in the external dataset. The Dice similarity coefficient (DSC) for individual delineation tasks and the primary tumor/metastasis classification accuracy were used as evaluation metrics. Additionally, a survival analysis using univariate Cox regression was performed comparing achieved group separation for manual and automated delineation, respectively.

Results: In the cross-validation experiment, delineation of all malignant lesions with the trained U-Net models achieves DSC of 0.885, 0.805, and 0.870 for primary tumor, LN metastases, and the union of both, respectively. In external testing, the DSC reaches 0.850, 0.724, and 0.823 for primary tumor, LN metastases, and the union of both, respectively. The voxel classification accuracy was 98.0% and 97.9% in cross-validation and external data, respectively. Univariate Cox analysis in the cross-validation and the external testing reveals that manually and automatically derived total MTVs are both highly prognostic with respect to overall survival, yielding essentially identical hazard ratios (HR) (${\text{HR}}_{\text{man}}=1.9$; $p<0.001$ vs. ${\text{HR}}_{\text{cnn}}=1.8$; $p<0.001$ in cross-validation and ${\text{HR}}_{\text{man}}=1.8$; $p=0.011$ vs. ${\text{HR}}_{\text{cnn}}=1.9$; $p=0.004$ in external testing).

Conclusion: To the best of our knowledge, this work presents the first CNN model for successful MTV delineation and lesion classification in HNC. In the vast majority of patients, the network performs satisfactory delineation and classification of primary tumor and lymph node metastases and only rarely requires more than minimal manual correction. It is thus able to massively facilitate study data evaluation in large patient groups and also does have clear potential for supervised clinical application.

Keywords: Convolutional neural network; FDG PET; HNC; Head and neck cancer; MTV; Metabolic tumor volume.

Fig. 1

Frequency distribution of the observed Dice coefficients (CNN vs. manual delineation/labeling) for primary tumor (left), LN metastases (middle), and the union of both (right) in cross-validation (top, $N=698$ patients) and external testing data (bottom, $N=181$ patients). The dashed vertical line indicates the location of a DSC = 0.7 threshold, a value which might be considered acceptable for practical use. The numbers to the left and to the right of the line specify the percentage of cases yielding a DSC below and above that threshold, respectively

Fig. 2

Correlation between manually and automatically derived total tumor burden (TTB: sum of primary tumor and LN metastases) in the cross-validation (left) and external testing (right) data. Note the difference in scale between the plots. Solid red points indicate outliers, defined as data points where the deviation of CNN from manual delineation exceeds the 99% percentile (i.e., the top 1%). These outliers were excluded from regression analysis. The red line represents the least squares fit of a straight line to the remaining data. The blue lines delineate the corresponding 95% prediction (tolerance) interval of expected scatter of individual data points around the regression line

Fig. 3

Manual and CNN-based delineations of primary tumor and lymph node metastases in 4 selected patients. Relevant transaxial PET/CT slices are shown (top: CT, bottom: PET). The dice coefficients (in the presented plane) for primary tumor (${\text{DSC}}_{\text{PT}}$), LN metastases (${\text{DSC}}_{\text{LN}}$), and their union (${\text{DSC}}_{\text{All}}$) are indicated. Patient A: oropharyngeal cancer with LN metastasis; patient B: oropharyngeal cancer with necrotic core; patient C: hypopharyngeal cancer with 2 LN metastases; patient D: cancer of oral cavity (not visible in this slice) exhibiting a low uptake LN metastasis

Fig. 4

Examples of CNN delineation errors in 4 selected patients. Relevant transaxial PET/CT slices are shown (top: CT, bottom: PET). The dice coefficients (in the presented plane) for primary tumor (${\text{DSC}}_{\text{PT}}$), LN metastases (${\text{DSC}}_{\text{LN}}$) and their union (${\text{DSC}}_{\text{All}}$) are indicated. Patient A with laryngeal cancer and multiple LN metastases (only one in plane): LN metastasis incorrectly classified as primary; patient B with oropharyngeal cancer and LN metastasis (both out of plane): CNN produced spurious LN metastasis ROI; patient C with nasopharyngeal cancer and LN metastases (out of plane): primary missed by CNN; patient D with oropharyngeal cancer and low and diffuse uptake LN metastasis: LN metastasis missed by CNN

Fig. 5

Kaplan-Meier curves with respect to OS in cross-validation (top) and external testing (bottom) data