Skip to main page content
U.S. flag

An official website of the United States government

Dot gov

The .gov means it’s official.
Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you’re on a federal government site.

Https

The site is secure.
The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

Access keys NCBI Homepage MyNCBI Homepage Main Content Main Navigation
Clinical Trial
. 2023 Jan 30:25:e40858.
doi: 10.2196/40858.

An Effective Test (EOmciSS) for Screening Older Adults With Mild Cognitive Impairment in a Community Setting: Development and Validation Study

Jingsong Wu  1   2 , Jingnan Tu  1   2 , Zhizhen Liu  2 , Lei Cao  1   2 , Youze He  1   2 , Jia Huang  1 , Jing Tao  3   4 , Mabel N K Wong  5   6 , Lidian Chen #  3   4 , Tatia M C Lee #  7 , Chetwyn C H Chan #  5
Affiliations
Clinical Trial

An Effective Test (EOmciSS) for Screening Older Adults With Mild Cognitive Impairment in a Community Setting: Development and Validation Study

Jingsong Wu et al. J Med Internet Res. .

Abstract

Background: Early detection of mild cognitive impairment (MCI) symptoms is an important step to its diagnosis and intervention. We developed a new screening test called "Efficient Online MCI Screening System" (EOmciSS) for use in community-dwelling older adults. It is a self-paced cognitive test to be completed within 10 minutes on tablets or smartphones in homes or care centers for older adults.

Objective: This study aims to test the validity of EOmciSS for identifying community-dwelling older adults with MCI risks.

Methods: Participants (N=827) completed EOmciSS and other screening tests for MCI. The psychometric properties tested were "subscale item difficulty," "discriminative index," "internal consistency," and "construct validity." We also tested between-group discrimination using the cross-validation method in an MCI group and a normal cognitive function (NCF) group.

Results: A total of 3 accuracy factors and 1 reaction time factor explained the structure of the 20 item factors. The difficulty level of accuracy factors (ie, "trail making," "clock drawing," "cube copying," "delayed recall") was 0.63-0.99, whereas that of the reaction time factor was 0.77-0.95. The discriminative index of the medium-to-high-difficulty item factors was 0.39-0.97. The internal consistency (Cronbach α) ranged from .41 (for few item factors) to .96. The training data set contained 9 item factors (CC-Acc1, P<.001; CD-Acc1, P=.07; CD-Acc2, P=.06; CD-Acc3, P<.001; TM-Acc4, P=.07; DR-Acc1, P=.03; RS, P=.06; DR-RT1, P=.02; and DR-RT2, P=.05) that were significant predictors for an MCI classification versus NCF classification. Depressive symptoms were identified as significant factors (P<.001) influencing the performance of participants, and were an integral part of our test system. Age (P=.15), number of years of education (P=.18), and proficiency in using an electronic device (P=.39) did not significantly influence the scores nor classification of participants. Application of the MCI/NCF cutoff score (7.90 out of 9.67) to the validation data set yielded an area under the curve of 0.912 (P<.001; 95% CI 0.868-0.955). The sensitivity was 84.9%, specificity was 85.1%, and the Youden index was 0.70.

Conclusions: EOmciSS was valid and reliable for identifying older adults with significant risks of MCI. Our results indicate that EOmciSS has higher sensitivity and specificity than those of the Computer-Administered Neuropsychological Screen for Mild Cognitive Impairment and the Computerized Cognitive Screen. The user interface, online operation, and self-paced format allowed the test system to be operated by older adults or their caregivers in different settings (eg, home or care centers for older adults). Depressive symptoms should be an integral part in future MCI screening systems because they influence the test performance and, hence, MCI risk.

Trial registration: Chinese Clinical Trial Registry ChiCTR2000039411; http://www.chictr.org.cn/showprojen.aspx?proj=62903.

Keywords: aging; community dwelling; digital assessment; digital health; mild cognitive impairment; older adults; screening test.

PubMed Disclaimer

Conflict of interest statement

Conflicts of Interest: This study was sponsored by Amity Brain Health and A.I. Technology Company Limited (which develops EOmciSS) and CCHC and TMCL are paid research consultants for these organizations.

Figures

Figure 1
Figure 1
Test contents of EOmciSS. The test contents of EOmciSS cover 5 subtypes with 20 test items. The subtypes are “depression,” “visual attention and mental flexibility,” “visuospatial and executive functions,” “memory,” and “cognitive processing speed.” EOmciSS: Efficient Online MCI Screening System; GDS-15: 15-item Geriatric Depression Scale.
Figure 2
Figure 2
EOmciSS workflow. The EOmciSS workflow starts from the “log-in system” to completion of the screening test with printing of the result. The entire workflow is to be completed within 10 minutes. EOmciSS: Efficient Online MCI Screening System; GDS-15: 15-item Geriatric Depression Scale.
Figure 3
Figure 3
Screenshot examples of EOmciSS. Screenshot examples of EOmciSS showing: (A) the login page, (B) a “depressive mood” item (GDS-15), (C) the demonstration page of a test item, (D) beginning of the “Trail Making” subscale, and (E) the printing page of test results. EOmciSS: Efficient Online MCI Screening System; GDS-15: 15-item Geriatric Depression Scale.
Figure 4
Figure 4
Path diagram showing the estimated parameter loadings from confirmatory factor analysis. A 4-factor structure is revealed on the accuracy, reaction time, and order factors for the 20 test items of EOmciSS. Acc: accuracy; CC: cube copying; CD: clock drawing; DR: delayed recall; EOmciSS: Efficient Online MCI Screening System; RT: reaction time; RS: reaction speed; TM: trail making.
Figure 5
Figure 5
ROC curves. ROC curves for the 9 EOmciSS item factors differentiating participants with MCI from those with NCF in the validation data set. AUC for the validation set was 0.912 (95% CI 0.868-0.955; P<.001). AUC: area under the curve; EOmciSS: Efficient Online MCI Screening System; MCI: mild cognitive impairment; NCF: normal cognitive function; ROC: receiver operating characteristic.
See this image and copyright information in PMC

References

    1. Livingston G, Huntley J, Sommerlad A, Ames D, Ballard C, Banerjee S, Brayne C, Burns A, Cohen-Mansfield J, Cooper C, Costafreda SG, Dias A, Fox N, Gitlin LN, Howard R, Kales HC, Kivimäki M, Larson EB, Ogunniyi A, Orgeta V, Ritchie K, Rockwood K, Sampson EL, Samus Q, Schneider LS, Selbæk G, Teri L, Mukadam N. Dementia prevention, intervention, and care: 2020 report of the Lancet Commission. The Lancet. 2020 Aug;396(10248):413–446. doi: 10.1016/s0140-6736(20)30367-6. - DOI - PMC - PubMed
    1. Prince M, Bryce R, Albanese E, Wimo A, Ribeiro W, Ferri CP. The global prevalence of dementia: a systematic review and metaanalysis. Alzheimers Dement. 2013 Jan;9(1):63–75.e2. doi: 10.1016/j.jalz.2012.11.007.S1552-5260(12)02531-9 - DOI - PubMed
    1. Zhou M, Wang H, Zeng X, Yin P, Zhu J, Chen W, Li X, Wang L, Wang L, Liu Y, Liu J, Zhang M, Qi J, Yu S, Afshin A, Gakidou E, Glenn S, Krish VS, Miller-Petrie MK, Mountjoy-Venning WC, Mullany EC, Redford SB, Liu H, Naghavi M, Hay SI, Wang L, Murray CJL, Liang X. Mortality, morbidity, and risk factors in China and its provinces, 1990–2017: a systematic analysis for the Global Burden of Disease Study 2017. The Lancet. 2019 Sep;394(10204):1145–1158. doi: 10.1016/s0140-6736(19)30427-1. - DOI - PMC - PubMed
    1. Ganguli M, Dodge HH, Shen C, DeKosky ST. Mild cognitive impairment, amnestic type: An epidemiologic study. Neurology. 2004 Jul 12;63(1):115–121. doi: 10.1212/01.wnl.0000132523.27540.81. - DOI - PubMed
    1. Patnode CD, Perdue LA, Rossom RC, Rushkin MC, Redmond N, Thomas RG, Lin JS. Screening for Cognitive Impairment in Older Adults: Updated Evidence Report and Systematic Review for the US Preventive Services Task Force. JAMA. 2020 Feb 25;323(8):764–785. doi: 10.1001/jama.2019.22258.2761650 - DOI - PubMed

Publication types