Reliability of a Computerized Neurocognitive Test in Baseline Concussion Testing of High School Athletes

Abstract

Objective: Baseline assessments using computerized neurocognitive tests are frequently used in the management of sport-related concussions. Such testing is often done on an annual basis in a community setting. Reliability is a fundamental test characteristic that should be established for such tests. Our study examined the test-retest reliability of a computerized neurocognitive test in high school athletes over 1 year.

Design: Repeated measures design.

Setting: Two American high schools.

Participants: High school athletes (N = 117) participating in American football or soccer during the 2011-2012 and 2012-2013 academic years.

Interventions: All study participants completed 2 baseline computerized neurocognitive tests taken 1 year apart at their respective schools. The test measures performance on 4 cognitive tasks: identification speed (Attention), detection speed (Processing Speed), one card learning accuracy (Learning), and one back speed (Working Memory).

Main outcome measures: Reliability was assessed by measuring the intraclass correlation coefficient (ICC) between the repeated measures of the 4 cognitive tasks. Pearson and Spearman correlation coefficients were calculated as a secondary outcome measure.

Results: The measure for identification speed performed best (ICC = 0.672; 95% confidence interval, 0.559-0.760) and the measure for one card learning accuracy performed worst (ICC = 0.401; 95% confidence interval, 0.237-0.542). All tests had marginal or low reliability.

Conclusions: In a population of high school athletes, computerized neurocognitive testing performed in a community setting demonstrated low to marginal test-retest reliability on baseline assessments 1 year apart. Further investigation should focus on (1) improving the reliability of individual tasks tested, (2) controlling for external factors that might affect test performance, and (3) identifying the ideal time interval to repeat baseline testing in high school athletes.

Clinical relevance: Computerized neurocognitive tests are used frequently in high school athletes, often within a model of baseline testing of asymptomatic individuals before the start of a sporting season. This study adds to the evidence that suggests in this population such testing may lack sufficient reliability to support clinical decision making.