Skip to main page content
U.S. flag

An official website of the United States government

Dot gov

The .gov means it’s official.
Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you’re on a federal government site.

Https

The site is secure.
The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

Access keys NCBI Homepage MyNCBI Homepage Main Content Main Navigation
Meta-Analysis
. 2021 Aug;36(8):1127-1147.
doi: 10.1002/gps.5499. Epub 2021 May 3.

Interrater agreement in dementia diagnosis: A systematic review and meta-analysis

Enzo Cerullo  1   2 , Terry J Quinn  2   3 , Jenny McCleery  4 , Elpida Vounzoulaki  5 , Nicola J Cooper  1   2 , Alex J Sutton  1   2
Affiliations
Meta-Analysis

Interrater agreement in dementia diagnosis: A systematic review and meta-analysis

Enzo Cerullo et al. Int J Geriatr Psychiatry. 2021 Aug.

Abstract

Objectives: Dementia remains a clinical diagnosis with a degree of subjective assessment and potential for interrater disagreement. We described interrater agreement of clinical dementia diagnosis for various diagnostic criteria.

Methods: We conducted a PROSPERO-registered (CRD42020168245) systematic review and meta-analysis. We searched multiple cross-disciplinary databases from inception until April 2020 for relevant papers, extracted data and described study quality in duplicate. Study quality was assessed using the Guidelines for Reporting Reliability and Agreement Studies. We used random-effects models to obtain summary estimates of interrater agreement using kappa and, where possible, Gwet's AC1/2 coefficients.

Results: We found 7577 titles and 22 eligible studies. Meta-analysis was possible for all-cause dementia using the Diagnostic and Statistical Manual of Mental Disorders third edition revised (DSM-III-R) criteria (kappa = 0.66, 95% CI = [0.53,0.78]), Alzheimer's disease using the National Institute of Neurological and Communicative Disorders and Stroke and the Alzheimer's disease and Related Disorders Association (NINCDS-ADRDA) criteria (kappa = 0.71, 95% CI = [0.65,0.77] for presence/absence and AC2 = 0.61, 95% CI = [0.53,0.70] when distinguishing probable/possible cases), and vascular dementia using the International Classification of Diseases version 10 (ICD-10) criteria kappa = 0.79 (95% CI = [0.70,0.87]). Data was more limited for other criteria and dementia types. AC1/2 coefficients generally indicated higher agreement. One study was rated as high quality.

Conclusions: Diagnostic criteria for clinical dementia may have good but imperfect agreement. This has important implications for clinical practice and research studies, which frequently assume these criteria are perfect tests, such as diagnostic test accuracy studies frequently conducted for biomarkers and neuropsychological tests, and for trials where incident dementia is the outcome.

Keywords: dementia; diagnosis; interrater agreement; meta-analysis; reliability; systematic review.

PubMed Disclaimer

References

REFERENCES

    1. Nelson PT, Braak H, Markesbery WR. Neuropathology and cognitive impairment in Alzheimer disease: a complex but coherent relationship. J Neuropathol Exp Neurol. 2009;68(1):1-14. https://doi.org/10.1097/NEN.0b013e3181919a48
    1. Savva GM, Wharton SB, Ince PG, Forster G, Matthews FE, Brayne C. Age, neuropathology, and dementia Published online. N Engl J Med. 2009;360:2302-2309. https://doi.org/10.1056/NEJMoa0806142
    1. Forlenza OV, Radanovic M, Talib LL, et al. Cerebrospinal fluid biomarkers in Alzheimer's disease: diagnostic accuracy and prediction of dementia Published online. Alzheimer's Dementia Diagnosis, Assess Dis Monit. 2015;1:455-463. https://doi.org/10.1016/j.dadm.2015.09.003
    1. Creavin ST, Wisniewski S, Noel-Storr AH, et al. Mini-mental state examination (MMSE) for the detection of dementia in clinically unevaluated people aged 65 and over in community and primary care populations. Cochrane Database Syst Rev. 2016;1:CD011145. https://doi.org/10.1002/14651858.CD011145.pub2
    1. Kottner J, Audigé L, Brorson S, et al. Guidelines for reporting reliability and agreement studies (GRRAS) were proposed. J Clin Epidemiol. 2011;64(1):96-106. https://doi.org/10.1016/j.jclinepi.2010.03.002

Publication types

LinkOut - more resources