Federated semi-supervised learning for COVID region segmentation in chest CT using multi-national data from China, Italy, Japan

Abstract

The recent outbreak of Coronavirus Disease 2019 (COVID-19) has led to urgent needs for reliable diagnosis and management of SARS-CoV-2 infection. The current guideline is using RT-PCR for testing. As a complimentary tool with diagnostic imaging, chest Computed Tomography (CT) has been shown to be able to reveal visual patterns characteristic for COVID-19, which has definite value at several stages during the disease course. To facilitate CT analysis, recent efforts have focused on computer-aided characterization and diagnosis with chest CT scan, which has shown promising results. However, domain shift of data across clinical data centers poses a serious challenge when deploying learning-based models. A common way to alleviate this issue is to fine-tune the model locally with the target domains local data and annotations. Unfortunately, the availability and quality of local annotations usually varies due to heterogeneity in equipment and distribution of medical resources across the globe. This impact may be pronounced in the detection of COVID-19, since the relevant patterns vary in size, shape, and texture. In this work, we attempt to find a solution for this challenge via federated and semi-supervised learning. A multi-national database consisting of 1704 scans from three countries is adopted to study the performance gap, when training a model with one dataset and applying it to another. Expert radiologists manually delineated 945 scans for COVID-19 findings. In handling the variability in both the data and annotations, a novel federated semi-supervised learning technique is proposed to fully utilize all available data (with or without annotations). Federated learning avoids the need for sensitive data-sharing, which makes it favorable for institutions and nations with strict regulatory policy on data privacy. Moreover, semi-supervision potentially reduces the annotation burden under a distributed setting. The proposed framework is shown to be effective compared to fully supervised scenarios with conventional data sharing instead of model weight sharing.

Keywords: COVID-19; Chest CT; Federated learning; Semi-supervision.

Graphical abstract

Fig. 1

Federated learning with privacy preserving in medical imaging. The central federated server communicates with clients from multi-national institutions by sharing weights and gradients of models, without exchanging any sensitive data information.

Fig. 2

The axial planes of chest CT scans from three different sites. Areas inside green contours represent COVID-19 affected regions annotated by radiologist. The appearance of the affected region identified as “infiltrates” range from diffused ground glass opacity (COVID, upper row) to focal nodules (lower row). (For interpretation of the references to colour in this figure legend, the reader is referred to the web version of this article.)

Fig. 3

The general architecture of fully convolutional network (FCN) for COVID-19 affected region in medical imaging. Dashed lines denote skip connections which feed earlier feature maps to later neural network layers.

Fig. 4

FL is to find global low-error space of ${\theta }_{\mathrm{global}}$ for client $A$ and $B$ after aggregation of “gradients” ${\Delta }_A$ and ${\Delta }_B$. The low-error space is defined by each client, which could correspond to high accuracy, or high model consistency for self-supervision.

Algorithm 1

Federated learning for COVID region segmentation using weighted federated averaging.

Algorithm 2

Self-supervised learning algorithm at clients with unlabeled data for COVID region segmentation.

Fig. 5

Visualizations of federated semi-supervised segmentation of COVID regions in 3D CT (from the testing set of unsupervised client Image_2). “Non-FL” indicates results from the model trained with Image_1 along, and “FL” denotes results from the model trained with federated semi-supervised learning on Image_1 and Image_2. The segmentation results using the proposed framework captures the ground truth shapes better and has less false positives.

Fig. 6

Accuracy (Dice’s score) comparison of different learning rates of un-/unsupervised clients.

Fig. 7

Accuracy (Dice’s score) comparison of different aggregation frequency (per 5, 10, 20, 40 rounds).

Fig. 8

Model performance on LIDC dataset, capturing solid nodule (1, 2); ground glass nodule (2, 3); and other abnormal patterns (4).

Fig. 9

Visualization of the COVID-19 affected region segmentation/prediction (red regions) together with lungs and airways (a) in 3D space, and (b,c,d) in different (axial, sagittal, coronal) planes of a raw CT image. Note that slice thickness is 5 mm (as compared with 0.8 mm in-plane), which is the case for most images in this work. . (For interpretation of the references to colour in this figure legend, the reader is referred to the web version of this article.)