Advanced machine learning in action: identification of intracranial hemorrhage on computed tomography scans of the head with clinical workflow integration

Abstract

Intracranial hemorrhage (ICH) requires prompt diagnosis to optimize patient outcomes. We hypothesized that machine learning algorithms could automatically analyze computed tomography (CT) of the head, prioritize radiology worklists and reduce time to diagnosis of ICH. 46,583 head CTs (~2 million images) acquired from 2007-2017 were collected from several facilities across Geisinger. A deep convolutional neural network was trained on 37,074 studies and subsequently evaluated on 9499 unseen studies. The predictive model was implemented prospectively for 3 months to re-prioritize "routine" head CT studies as "stat" on realtime radiology worklists if an ICH was detected. Time to diagnosis was compared between the re-prioritized "stat" and "routine" studies. A neuroradiologist blinded to the study reviewed false positive studies to determine whether the dictating radiologist overlooked ICH. The model achieved an area under the ROC curve of 0.846 (0.837-0.856). During implementation, 94 of 347 "routine" studies were re-prioritized to "stat", and 60/94 had ICH identified by the radiologist. Five new cases of ICH were identified, and median time to diagnosis was significantly reduced (p < 0.0001) from 512 to 19 min. In particular, one outpatient with vague symptoms on anti-coagulation was found to have an ICH which was treated promptly with reversal of anticoagulation, resulting in a good clinical outcome. Of the 34 false positives, the blinded over-reader identified four probable ICH cases overlooked in original interpretation. In conclusion, an artificial intelligence algorithm can prioritize radiology worklists to reduce time to diagnosis of new outpatient ICH by 96% and may also identify subtle ICH overlooked by radiologists. This demonstrates the positive impact of advanced machine learning in radiology workflow optimization.

Keywords: Brain imaging; Computed tomography.

Fig. 1

Proposed CNN architecture for ICH detection. This architecture has five convolutional and two fully connected layers

Fig. 2

ROC curve of the ICH detection algorithm on the testing dataset. The magenta circle illustrates the operating point. AUC of the ROC is 0.846

Fig. 3

Examples of true positive ICH cases detected by the algorithm. a Outpatient with right thalamic hemorrhage. b Outpatient with subarachnoid hemorrhage in the left frontal convexity. c Inpatient with acute intraventricular hemorrhage within the third ventricle. d Outpatient with right basal ganglia/external capsular hemorrhage. e Inpatient with chronic subdural hemorrhage. f Outpatient with large left frontoparietal intraparenchymal hemorrhage. g Inpatient with intraparenchymal hemorrhagic contusion in the right temporal lobe. h Inpatient with large intraparenchymal hematoma. i Outpatient with bilateral subdural hemorrhages. j Outpatient with subdural hemorrhage

Fig. 4

Clinical implementation of the ICH detection algorithm as a radiology workflow optimization tool

Fig. 5

Representative head CT image slices associated with case #1 (a) and case #2 (b)