Deep Learning-Assisted Diagnosis of Cerebral Aneurysms Using the HeadXNet Model

Abstract

Importance: Deep learning has the potential to augment clinician performance in medical imaging interpretation and reduce time to diagnosis through automated segmentation. Few studies to date have explored this topic.

Objective: To develop and apply a neural network segmentation model (the HeadXNet model) capable of generating precise voxel-by-voxel predictions of intracranial aneurysms on head computed tomographic angiography (CTA) imaging to augment clinicians' intracranial aneurysm diagnostic performance.

Design, setting, and participants: In this diagnostic study, a 3-dimensional convolutional neural network architecture was developed using a training set of 611 head CTA examinations to generate aneurysm segmentations. Segmentation outputs from this support model on a test set of 115 examinations were provided to clinicians. Between August 13, 2018, and October 4, 2018, 8 clinicians diagnosed the presence of aneurysm on the test set, both with and without model augmentation, in a crossover design using randomized order and a 14-day washout period. Head and neck examinations performed between January 3, 2003, and May 31, 2017, at a single academic medical center were used to train, validate, and test the model. Examinations positive for aneurysm had at least 1 clinically significant, nonruptured intracranial aneurysm. Examinations with hemorrhage, ruptured aneurysm, posttraumatic or infectious pseudoaneurysm, arteriovenous malformation, surgical clips, coils, catheters, or other surgical hardware were excluded. All other CTA examinations were considered controls.

Main outcomes and measures: Sensitivity, specificity, accuracy, time, and interrater agreement were measured. Metrics for clinician performance with and without model augmentation were compared.

Results: The data set contained 818 examinations from 662 unique patients with 328 CTA examinations (40.1%) containing at least 1 intracranial aneurysm and 490 examinations (59.9%) without intracranial aneurysms. The 8 clinicians reading the test set ranged in experience from 2 to 12 years. Augmenting clinicians with artificial intelligence-produced segmentation predictions resulted in clinicians achieving statistically significant improvements in sensitivity, accuracy, and interrater agreement when compared with no augmentation. The clinicians' mean sensitivity increased by 0.059 (95% CI, 0.028-0.091; adjusted P = .01), mean accuracy increased by 0.038 (95% CI, 0.014-0.062; adjusted P = .02), and mean interrater agreement (Fleiss κ) increased by 0.060, from 0.799 to 0.859 (adjusted P = .05). There was no statistically significant change in mean specificity (0.016; 95% CI, -0.010 to 0.041; adjusted P = .16) and time to diagnosis (5.71 seconds; 95% CI, 7.22-18.63 seconds; adjusted P = .19).

Conclusions and relevance: The deep learning model developed successfully detected clinically significant intracranial aneurysms on CTA. This suggests that integration of an artificial intelligence-assisted diagnostic model may augment clinician performance with dependable and accurate predictions and thereby optimize patient care.

Figure 1.. Study Design

A, Crossover study design. Clinicians were divided into 2 groups to perform reads with and without model augmentation in random order, with a 2-week washout period between. B, Unaugmented read, with original CTA scan in axial, coronal, and sagittal view. C, Augmented read, with model segmentation overlay on CTA in axial, coronal, and sagittal view. Readers had the option to toggle overlays off and view the scan as shown in B. AI indicates artificial intelligence; CTA, computed tomographic angiography.

Figure 2.. Data Set Selection Flow Diagram and Patient Demographics

Of 9455 computed tomography angiogram (CTA) examinations performed between 2003 and 2017 at Stanford University Medical Center, 818 were selected according to an exclusion criteria validated by a board-certified neuroradiologist. These examinations were split into the training set, development set, and test set to be used for training models, selecting the best model, and assessing the selected model, respectively.

Figure 3.. Change in Individual Clinicians' Performance Metric

Horizontal lines depict the change in performance metric for each clinician with and without model augmentation. The orange dot represents performance without model, and the blue dot represents performance with model augmentation.