Skip to main page content
U.S. flag

An official website of the United States government

Dot gov

The .gov means it’s official.
Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you’re on a federal government site.

Https

The site is secure.
The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

Access keys NCBI Homepage MyNCBI Homepage Main Content Main Navigation
. 2022 Nov;36(8):2153-2167.
doi: 10.1080/13854046.2021.1954244. Epub 2021 Jul 26.

Initial investigation of test-retest reliability of home-to-home teleneuropsychological assessment in healthy, English-speaking adults

Joshua T Fox-Fuller  1   2 , Julie Ngo  1 , Celina F Pluim  1   2 , Rini I Kaplan  1 , Dong-Ho Kim  1 , Juliana A U Anzai  1 , Defne Yucebas  1 , Soibifaa M Briggs  1 , Paula A Aduen  2 , Alice Cronin-Golomb  1 , Yakeel T Quiroz  2   3
Affiliations

Initial investigation of test-retest reliability of home-to-home teleneuropsychological assessment in healthy, English-speaking adults

Joshua T Fox-Fuller et al. Clin Neuropsychol. 2022 Nov.

Abstract

Prior teleneuropsychological research has assessed the reliability between in-person and remote administration of cognitive assessments. Few, if any, studies have examined the test-retest reliability of cognitive assessments conducted in sequential clinic-to-home or home-to-home teleneuropsychological evaluations - a critical issue given the state of clinical practice during the COVID-19 pandemic. This study examined this key psychometric question for several cognitive tests administered over repeated videoconferencing visits 4-6 months apart in a sample of healthy English-speaking adults.

A total of 44 participants (ages 18-75) completed baseline and follow-up cognitive testing 4-6 months apart. Testing was conducted in a home-to-home setting over HIPAA-compliant videoconferencing meetings on participants' audio-visual enabled laptop or desktop computers. The following measures were repeated at both virtual visits: the Controlled Oral Word Association Test (FAS), Category Fluency (Animals), and Digit Span Forward and Backward from the Wechsler Adult Intelligence Scale, Fourth Edition. Intraclass correlation coefficients (ICC), Pearson correlations, root mean square difference (RMSD), and concordance correlation coefficients (CCC) were calculated as test-retest reliability metrics, and practice effects were assessed using paired-samples t-tests.

Some tests exhibited small practice effects, and test-retest reliability was marginal or worse for all measures except FAS, which had adequate reliability (based on ICC and r). Reliability estimates with RMSD suggested that change within +/- 1 SD on these measures may reflect typical test-retest variability.

The included cognitive measures exhibited questionable reliability over repeated home-to-home videoconferencing evaluations. Future teleneuropsychology test-retest reliability research is needed with larger, more diverse samples and in clinical populations.

PubMed Disclaimer

Conflict of interest statement

Disclosure statement

YTQ has served as a paid consultant for Biogen (Not related to the content in the manuscript). All co-authors report no conflicts of interest.

Figures

Figure 1.
Figure 1.
Bland–Altman plot of agreement between repeated administrations of cognitive measures over videoconferencing. Shown is the agreement (change score relative to mean of Time 1 and 2) for each participant on (A) animal fluency, (B) FAS, (C) WAIS-IV Digit Span Forward, and (D) WAIS-IV Digit Span Backward using demographically-adjusted and scalded scores. The solid black lines represent the sample-derived mean change values, whereas thin black lines represent +/− 1 SD from a mean of 0. The dashed and solid line represents a mean of 0. Lastly, the dashed lines represent the sample-derived 95% confidence interval on the measure. FAS= letter fluency from the Controlled Oral Word Association Test; WAIS-IV = Wechsler Adult Intelligence Scale, Fourth Edition.
See this image and copyright information in PMC

References

    1. Anderson M, Perrin A, Jiang J, & Kumar M (2019). 10% of Americans don’t use the internet. Who are they? Pew Research Center. https://www.pewresearch.org/fact-tank/2019/04/22/some-americans-dont-use...
    1. Appleman ER, O’Connor MK, Boucher SJ, Rostami R, Sullivan SK, Migliorini R, & Kraft M (2021). Teleneuropsychology clinic development and patient satisfaction. The Clinical Neuropsychologist, 35(4), 819–819. 10.1080/13854046.2020.1871515 - DOI - PubMed
    1. Barchard KA (2012). Examining the reliability of interval level data using root mean square differences and concordance correlation coefficients. Psychological Methods, 17(2), 294–308. 10.1037/a0023351 - DOI - PubMed
    1. Bilder RM, Postal KS, Barisa M, Aase DM, Cullum CM, Gillaspy SR, Harder L, Kanter G, Lanca M, Lechuga DM, Morgan JM, Most R, Puente AE, Salinas CM, & Woodhouse J (2020). Inter organizational practice committee recommendations/Guidance for teleneuropsychology in response to the COVID-19 Pandemic†. Archives of Clinical Neuropsychology: The Official Journal of the National Academy of Neuropsychologists, 35(6), 647–659. 10.1093/arclin/acaa046 - DOI - PMC - PubMed
    1. Binder LM, Iverson GL, & Brooks BL (2009). To Err is Human: “Abnormal” neuropsychological scores and variability are common in healthy adults. Archives of Clinical Neuropsychology, 24(1), 31–46. 10.1093/arclin/acn001 - DOI - PubMed

Publication types