Advancing COVID-19 Diagnosis with Privacy-Preserving Collaboration in Artificial Intelligence

Abstract

Artificial intelligence (AI) provides a promising substitution for streamlining COVID-19 diagnoses. However, concerns surrounding security and trustworthiness impede the collection of large-scale representative medical data, posing a considerable challenge for training a well-generalised model in clinical practices. To address this, we launch the Unified CT-COVID AI Diagnostic Initiative (UCADI), where the AI model can be distributedly trained and independently executed at each host institution under a federated learning framework (FL) without data sharing. Here we show that our FL model outperformed all the local models by a large yield (test sensitivity /specificity in China: 0.973/0.951, in the UK: 0.730/0.942), achieving comparable performance with a panel of professional radiologists. We further evaluated the model on the hold-out (collected from another two hospitals leaving out the FL) and heterogeneous (acquired with contrast materials) data, provided visual explanations for decisions made by the model, and analysed the trade-offs between the model performance and the communication costs in the federated training process. Our study is based on 9,573 chest computed tomography scans (CTs) from 3,336 patients collected from 23 hospitals located in China and the UK. Collectively, our work advanced the prospects of utilising federated learning for privacy-preserving AI in digital health.

Fig. 1 |. Conceptual architecture of UCADI.

The participants first download and train the 3D CNN models based on the data of local cohorts. The trained model parameters are then encrypted and transmitted back to the server. Finally, the server produces the federated model via aggregating the contributions from each participant while without explicit access to the parameters.

Fig. 2 |. Deployment and workflow of UCADI participants.

a, Data. Construct a local dataset based on the high-quality, well-annotated and anonymised CTs. b, Flow. The backbone of the 3D DenseNet model mainly consists of six 3D dense blocks (in green), two 3D transmit blocks (in white), and an output layer (in grey). CTs of each case are converted into a (16,128,128) tensor after adaptive sampling, decentralisation and trilinear interpolation, then feed into the 3D CNN model for pneumonia classification. c, Process. During training, the model outputs are used to calculate the weighted cross entropy to update the network parameters. While testing, five independent predictions of each case are incorporated to report the predictive diagnostic results.

Fig. 3 |. Overview of CTs.

a, Radiological features correlated with COVID-19 pneumonia cases: ground glass opacity, interlobular septal thickening and consolidation (from left to right). b, Other non-COVID-19 cases, incl. healthy, other viral and bacterial pneumonia. c, Localised class-discriminative regions generated by GradCAM (in the heatmap) and annotated by the professional radiologists (circled in red), for COVID-19 cases.

Fig. 4 |. COVID-19 pneumonia identification performance of 3D CNN models trained on four different data resources (Main Campus, Optical Valley, Sino-French and NCCID) individually and federatively.

a, Receiver Operating Characteristic (ROC) curves when the models are tested on the data from China, in comparison with six professional radiologists, b, ROC curves of the CNN models tested on the data from the UK, c, Numeric results of the test sensitivity, specificity and area under the curve (AUC, with 95% confidence intervals and p-values)

Fig. 5 |. Trade-off on the performance and communication cost in federated training.

a, Relationships between transmission expense and model generalisation, b, Estimated time spent at different communication/synchronisation intervals. The statistics is measured based on a joint FL training of two clients. Each client has 200 CTs and 100 CTs for training and testing, respectively. The client’s software infrastructure is a single-core of GPU (NVIDIA GTX 1080Ti) and a CPU (Xeon(R) CPU E5–2660 v4 @ 2.00GHz). The bandwidth for transmission is around 7.2Mb/s (900KB/s), which is the average broadband speed.