Quantification of pulmonary involvement in COVID-19 pneumonia by means of a cascade of two U-nets: training and assessment on multiple datasets using different annotation criteria

Abstract

Purpose: This study aims at exploiting artificial intelligence (AI) for the identification, segmentation and quantification of COVID-19 pulmonary lesions. The limited data availability and the annotation quality are relevant factors in training AI-methods. We investigated the effects of using multiple datasets, heterogeneously populated and annotated according to different criteria.

Methods: We developed an automated analysis pipeline, the LungQuant system, based on a cascade of two U-nets. The first one (U-net[Formula: see text]) is devoted to the identification of the lung parenchyma; the second one (U-net[Formula: see text]) acts on a bounding box enclosing the segmented lungs to identify the areas affected by COVID-19 lesions. Different public datasets were used to train the U-nets and to evaluate their segmentation performances, which have been quantified in terms of the Dice Similarity Coefficients. The accuracy in predicting the CT-Severity Score (CT-SS) of the LungQuant system has been also evaluated.

Results: Both the volumetric DSC (vDSC) and the accuracy showed a dependency on the annotation quality of the released data samples. On an independent dataset (COVID-19-CT-Seg), both the vDSC and the surface DSC (sDSC) were measured between the masks predicted by LungQuant system and the reference ones. The vDSC (sDSC) values of 0.95±0.01 and 0.66±0.13 (0.95±0.02 and 0.76±0.18, with 5 mm tolerance) were obtained for the segmentation of lungs and COVID-19 lesions, respectively. The system achieved an accuracy of 90% in CT-SS identification on this benchmark dataset.

Conclusion: We analysed the impact of using data samples with different annotation criteria in training an AI-based quantification system for pulmonary involvement in COVID-19 pneumonia. In terms of vDSC measures, the U-net segmentation strongly depends on the quality of the lesion annotations. Nevertheless, the CT-SS can be accurately predicted on independent test sets, demonstrating the satisfactory generalization ability of the LungQuant.

Keywords: COVID-19; Chest Computed Tomography; Ground-glass opacities; Machine Learning; Segmentation; U-net.

Fig. 1

A summary of the whole analysis pipeline: the input CT scans are used to train U-net${}_1$, which is devoted to lung segmentation; its output is refined by a morphology-based method. A bounding box containing the segmented lungs is made and applied to all CT scans for training U-net${}_2$, which is devoted to COVID-19 lesion segmentation. Finally, the output of U-net${}_2$ is the definitive COVID-19 lesion mask, whereas the definitive lung mask is obtained as the union between the outputs of U-net${}_1$ and U-net${}_2$. The ratio between the COVID-19 lesion mask and the lung mask provides the CT-SS for each patient

Fig. 2

U-net scheme: the neural network is made of 6 levels of depth. In the compression path (left), the input is processed through convolutions, activation layers (ReLu) and instance normalization layers, while in the decompression one (right), in addition to those already mentioned, 3D Transpose Convolution (de-convolution) layers are also introduced

Fig. 3

On the rows: three axial slices of the first CT scan on the COVID-19-CT-Seg test dataset (coronacases001.nii) are shown. On the columns: original images (left); overlays between the predicted and the reference lung (centre) and COVID-19 lesion (right) masks. The reference masks are in green, while the predicted ones, obtained by the LungQuant system integrating U-net${}_2^{90\%}$,are in blue

Fig. 4

Estimated percentages P of affected lung volume versus the ground truth percentages, as obtained by the LungQuant system integrating U-net${}_2^{60\%}$ (left) and U-net${}_2^{90\%}$ (right). The grey areas in the plot backgrounds guide the eye to recognize the CT-SS values assigned to each value of P (from left to right: CT-SS $=$ 1, CT-SS $=$ 2, CT-SS $=$ 3)