Diagnostic accuracy of tablet-based software for the detection of concussion

Abstract

Despite the high prevalence of traumatic brain injuries (TBI), there are few rapid and straightforward tests to improve its assessment. To this end, we developed a tablet-based software battery ("BrainCheck") for concussion detection that is well suited to sports, emergency department, and clinical settings. This article is a study of the diagnostic accuracy of BrainCheck. We administered BrainCheck to 30 TBI patients and 30 pain-matched controls at a hospital Emergency Department (ED), and 538 healthy individuals at 10 control test sites. We compared the results of the tablet-based assessment against physician diagnoses derived from brain scans, clinical examination, and the SCAT3 test, a traditional measure of TBI. We found consistent distributions of normative data and high test-retest reliability. Based on these assessments, we defined a composite score that distinguishes TBI from non-TBI individuals with high sensitivity (83%) and specificity (87%). We conclude that our testing application provides a rapid, portable testing method for TBI.

Fig 1. Screenshots of the 6 assessments in the BrainCheck battery.

Fig 2. Normative data.

Each panel shows data from the indicated assessment. Only data from healthy individuals was used to create these histograms.

Fig 3. Age dependence of the BrainCheck assessments.

For each battery, data shows the mean value of the indicated age ranges. Error bars indicate standard error of the mean, computed by bootstrapping.

Fig 4

Performance on BrainCheck assessments by (A) gender and (B) socioeconomic status. As explained in the text, assessments performed at different university test sites were used as a rough measure of the effect of socioeconomic status. Error bars represent standard error of the mean, computed by bootstrapping.

Fig 5. Test-retest reliability.

In all panels, each datapoint represents an individual who took the same assessment on two different dates. Black lines represent linear fits to the data. r-values for the fits are shown in the legend of each panel.

Fig 6. Individual metrics differ for concussed and healthy individuals.

For each assessment, normative histograms for healthy (blue) or concussed (red) individuals are shown.

Fig 7. Sensitivity and specificity of individual assessments.

For each assessment the sensitivity (true positive rate; red) and specificity (true negative rate; blue) are plotted as a function of the threshold for discriminating concussed from healthy individuals.

Fig 8. A composite score distinguishes concussed and healthy individuals.

(A) Distribution of the composite score for healthy (blue) and concussed (red) individuals. (B) Sensitivity (red) and specificity (blue) of a test based on the composite score plotted as a function of the threshold for identifying concussed individuals.

Fig 9. A test for malingering shows no dependence on cognitive performance.

(A) Screenshot of the malingering test. (B) Normative data for the test (C) Scores on the malingering test plotted by age. (D) Data for individuals who took the malingering test twice separated by at least one week. The score on the first trial is plotted against the score on the second. (E) Distribution of scores on the malingering test for healthy (blue) and concussed (red) individuals. (F) Specificity (blue) and sensitivity (red) of a test to distinguish concussed and healthy individuals based on the malingering test.