Delirium Item Bank: Utilization to Evaluate and Create Delirium Instruments

Abstract

Introduction: The large number of heterogeneous instruments in active use for identification of delirium prevents direct comparison of studies and the ability to combine results. In a recent systematic review we performed, we recommended four commonly used and well-validated instruments and subsequently harmonized them using advanced psychometric methods to develop an item bank, the Delirium Item Bank (DEL-IB). The goal of the present study was to find optimal cut-points on four existing instruments and to demonstrate use of the DEL-IB to create new instruments.

Methods: We used a secondary analysis and simulation study based on data from three previous studies of hospitalized older adults (age 65+ years) in the USA, Ireland, and Belgium. The combined dataset included 600 participants, contributing 1,623 delirium assessments, and an overall incidence of delirium of about 22%. The measurements included the Diagnostic and Statistical Manual of Mental Disorders, 5th Edition diagnostic criteria for delirium, Confusion Assessment Method (long form and short form), Delirium Observation Screening Scale, Delirium Rating Scale-Revised-98 (total and severity scores), and Memorial Delirium Assessment Scale (MDAS).

Results: We identified different cut-points for each existing instrument to optimize sensitivity or specificity, and compared instrument performance at each cut-point to the author-defined cut-point. For instance, the cut-point on the MDAS that maximizes both sensitivity and specificity was at a sum score of 6 yielding 89% sensitivity and 79% specificity. We then created four new example instruments (two short forms and two long forms) and evaluated their performance characteristics. In the first example short form instrument, the cut-point that maximizes sensitivity and specificity was at a sum score of 3 yielding 90% sensitivity, 81% specificity, 30% positive predictive value, and 99% negative predictive value.

Discussion/conclusion: We used the DEL-IB to better understand the psychometric performance of widely used delirium identification instruments and scorings, and also demonstrated its use to create new instruments. Ultimately, we hope that the DEL-IB might be used to create optimized delirium identification instruments and to spur the development of a unified approach to identify delirium.

Keywords: Delirium; Item bank; Item response theory; Measurement; Psychometrics.

Fig. 1.

ROC curves for each delirium identification instrument compared to DSM-5 criteria. The ROC for each of the instruments and their different scorings, plus the latent trait (propensity to delirium) is shown. Each curve displays the instrument AUC. The large dot on each curve is the author-described cut-point on each instrument (except for the latent trait curve where the dot is at the cut-point that maximizes Youden’s J statistic). The CAM short form and long form each use the same diagnostic algorithm to identify delirium. CAM, Confusion Assessment Method; DOSS, Delirium Observation Screening Scale; DRS-R-98, Delirium Rating Scale-Revised-98; MDAS, Memorial Delirium Assessment Scale; AUC, area under the curve.

Fig. 2.

ROC curves for each example instrument compared to DSM-5 criteria. The ROC for each of the example instruments, plus the latent trait (propensity to delirium) is shown. Each curve displays the instrument AUC.