Computational Approaches for Acute Traumatic Brain Injury Image Recognition

Abstract

In recent years, there have been major advances in deep learning algorithms for image recognition in traumatic brain injury (TBI). Interest in this area has increased due to the potential for greater objectivity, reduced interpretation times and, ultimately, higher accuracy. Triage algorithms that can re-order radiological reading queues have been developed, using classification to prioritize exams with suspected critical findings. Localization models move a step further to capture more granular information such as the location and, in some cases, size and subtype, of intracranial hematomas that could aid in neurosurgical management decisions. In addition to the potential to improve the clinical management of TBI patients, the use of algorithms for the interpretation of medical images may play a transformative role in enabling the integration of medical images into precision medicine. Acute TBI is one practical example that can illustrate the application of deep learning to medical imaging. This review provides an overview of computational approaches that have been proposed for the detection and characterization of acute TBI imaging abnormalities, including intracranial hemorrhage, skull fractures, intracranial mass effect, and stroke.

Keywords: artificial intelligence; deep learning; evidence-based medicine; image recognition; precision medicine; traumatic brain injury.

Figure 1

Intracranial hemorrhage subtypes and their frequencies among mild TBI patients enrolled in the TRACK-TBI prospective longitudinal study of acute TBI (39). (A) Illustrates the various subtypes of intracranial hemorrhage, with red arrows indicating the abnormal lesion. (B) Shows the frequencies of each subtype of hemorrhage. SAH is the most commonly observed subtype, followed by SDH and contusion. Although the overall average incidence of “complicated” mild TBI (mild TBI with presence of acute intracranial hemorrhage on head CT) in the U.S. is lower in clinical practice than in TRACK-TBI (40), the relative distribution of hemorrhage subtypes within mild TBI is likely similar (39).

Figure 2

Rule-based algorithms. (A) Illustrates one possible workflow for a rule-based intracranial hemorrhage detection model. Application of thresholding and connectivity is a common way to identify regions of hematoma (8). (B) Shows examples of false positive errors from rule-based models in which streak artifacts are incorrectly labeled as regions of intracranial hemorrhage. This can result if regions of hemorrhage are obtained by thresholding for brighter pixels, which is a common strategy in rule-based models. Red indicates regions of algorithmic predictions (9).

Figure 3

Deep learning algorithms. (A) Is a schematic representation of an artificial neural network. n indicates the number of hidden layers. A network with n = 1, as seen in the figure, is the most basic and shallow form of a neural network. n can be increased to deepen the network and broaden its representation capacity. Increasing n results in the ability of the network to model tasks of increasing complexity, but also requires more training data to avoid overfitting (in which the model merely exploits the extra variables to achieve high performance on a specific training dataset, but fails to perform similarly on data outside of the training dataset). (B) Illustrates a convolutional neural network. Convolutional filters (in gold) are applied across the image to extract features, followed by pooling filters (in silver) that reduce feature map dimensionality. Most CNNs use multiple convolutional and pooling layers. The end layers include the fully connected and output layers.

Figure 4

A schematic of three labeling strategies in order of increasing granularity. Red indicates the label(s). (A) Demonstrates examination labels, in which an entire exam is annotated as “positive” or “negative” for a given pathology. (B) Demonstrates image labels, where each image in a stack is annotated as “positive” or “negative” for a pathology. (C) Demonstrates pixel labels, where all pixels in the exam are labeled as “positive” or “negative” (29).

Figure 5

A schematic representation of (A) classification and (B) segmentation algorithmic outputs. In (B), red regions indicate areas of acute intracranial hemorrhage designated by the algorithm (29).

Figure 6

A schematic of one way in which two sequential algorithms can be integrated into triage workflow for exam classification (23).

Figure 7

Incorporation of a reprioritization, or triage, algorithm into the radiological workflow in acute TBI (24). (A) Illustrates the broad range of pathological findings represented in the head CT training data, and how they were classified into non-urgent and urgent categories. (B) Shows the typical order in which critical (orange) and non-critical (gray) head CT exams would be interpreted by a radiologist before (left graph) and after (right graph) reprioritization by a deep learning algorithm. The gray and orange dots represent discrete CT exams, while the shaded regions are the smoothed exam frequency distributions. (C) Is a schematic representation of the algorithm's prioritization process.

Figure 8

Examples of accurate and erroneous predictions of abnormalities on head CT in acute TBI patients by a deep learning algorithm (26). Although individual images are shown, the model classifies abnormalities at the head CT exam level. All images under Accurate Predictions (A–I) have arrows added to indicate the abnormal lesion. All images under Erroneous Predictions (J–L) have arrows added to indicate the erroneous lesion predictions.

Figure 9

An example of a deep learning algorithm to localize intracranial hemorrhage and predict its subtype (28). Figure depicts the algorithmic output for a single head CT exam. (A) Demonstrates the probability determined by the algorithm for presence of each subtype of intracranial hemorrhage on each image. A 40% probability was designated as the minimum probability threshold to indicate the presence of a hemorrhage subtype on an image. The legend to the right shows the intracranial hemorrhage subtype that corresponds to each color. The boxes around the slice numbers indicate the example slices shown in the row of images below the graph, with the colors of the boxes indicating hemorrhage subtype(s) present on each image. Colored arrows on the images indicate the general prediction location and hemorrhage subtype. In (B), a probabilistic heatmap is superimposed on the brain to indicate a more specific region of prediction. (C) Displays prediction bases, which are the most relevant training images for specific hemorrhage subtypes. These can be examined by human practitioners to gain insight into the main drivers, or “rationale,” behind the algorithm's predictions, thereby increasing explainability.

Figure 10

Examples of intracranial hemorrhage detection, classification, and segmentation by a convolutional neural network (CNN) (29). (Left) Binary segmentations, in which the model indicates the presence or absence of intracranial hemorrhage only. (A,D,G,J). The first column shows the original head CT images. (B,E,H,K) The middle column shows the same images with orange shading of pixel-level probabilities >0.5 for intracranial hemorrhage as determined by the CNN. (C,F,I,L) The third column shows the original images with a blue outline drawn by an expert neuroradiologist around all areas of intracranial hemorrhage. (Right) Multiclass segmentations, in which the model not only detects intracranial hemorrhage but additionally indicates the hemorrhage subtype. (A,D,G,J,M,P) The first column shows original head CT images. (B,E,H,K,N,Q) The second column shows the algorithm's predictions. Each subtype is indicated by a different color, where subdural hematoma is green, brain contusion is purple, and subarachnoid hemorrhage is red. (C,F,I,L,O,R) The third column shows the “ground truth” labels drawn by expert neuroradiologists.