Deep learning-based artificial intelligence model to assist thyroid nodule diagnosis and management: a multicentre diagnostic study

Abstract

Background: Strategies for integrating artificial intelligence (AI) into thyroid nodule management require additional development and testing. We developed a deep-learning AI model (ThyNet) to differentiate between malignant tumours and benign thyroid nodules and aimed to investigate how ThyNet could help radiologists improve diagnostic performance and avoid unnecessary fine needle aspiration.

Methods: ThyNet was developed and trained on 18 049 images of 8339 patients (training set) from two hospitals (the First Affiliated Hospital of Sun Yat-sen University, Guangzhou, China, and Sun Yat-sen University Cancer Center, Guangzhou, China) and tested on 4305 images of 2775 patients (total test set) from seven hospitals (the First Affiliated Hospital of Guangzhou University of Chinese Medicine, Guangzhou, China; the Sixth Affiliated Hospital of Sun Yat-sen University, Guangzhou, China; the Guangzhou Army General Hospital, Guangzhou, China; the Third Affiliated Hospital of Sun Yat-sen University, Guangzhou, China; the First Affiliated Hospital of Sun Yat-sen University; Sun Yat-sen University Cancer Center; and the First Affiliated Hospital of Guangxi Medical University, Nanning, China) in three stages. All nodules in the training and total test set were pathologically confirmed. The diagnostic performance of ThyNet was first compared with 12 radiologists (test set A); a ThyNet-assisted strategy, in which ThyNet assisted diagnoses made by radiologists, was developed to improve diagnostic performance of radiologists using images (test set B); the ThyNet assisted strategy was then tested in a real-world clinical setting (using images and videos; test set C). In a simulated scenario, the number of unnecessary fine needle aspirations avoided by ThyNet-assisted strategy was calculated.

Findings: The area under the receiver operating characteristic curve (AUROC) for accurate diagnosis of ThyNet (0·922 [95% CI 0·910-0·934]) was significantly higher than that of the radiologists (0·839 [0·834-0·844]; p<0·0001). Furthermore, ThyNet-assisted strategy improved the pooled AUROC of the radiologists from 0·837 (0·832-0·842) when diagnosing without ThyNet to 0·875 (0·871-0·880; p<0·0001) with ThyNet for reviewing images, and from 0·862 (0·851-0·872) to 0·873 (0·863-0·883; p<0·0001) in the clinical test, which used images and videos. In the simulated scenario, the number of fine needle aspirations decreased from 61·9% to 35·2% using the ThyNet-assisted strategy, while missed malignancy decreased from 18·9% to 17·0%.

Interpretation: The ThyNet-assisted strategy can significantly improve the diagnostic performance of radiologists and help reduce unnecessary fine needle aspirations for thyroid nodules.

Funding: National Natural Science Foundation of China and Guangzhou Science and Technology Project.