Skip to main page content
U.S. flag

An official website of the United States government

Dot gov

The .gov means it’s official.
Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you’re on a federal government site.

Https

The site is secure.
The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

Access keys NCBI Homepage MyNCBI Homepage Main Content Main Navigation
. 2021 Jul 1;4(7):e2115707.
doi: 10.1001/jamanetworkopen.2021.15707.

Development and Validation of Computerized Adaptive Assessment Tools for the Measurement of Posttraumatic Stress Disorder Among US Military Veterans

Lisa A Brenner  1   2   3 , Lisa M Betthauser  1   2 , Molly Penzenik  1 , Anne Germain  4 , Jin Jun Li  5   6 , Ishanu Chattopadhyay  5   7   8 , Ellen Frank  4 , David J Kupfer  4 , Robert D Gibbons  5   7   9
Affiliations

Development and Validation of Computerized Adaptive Assessment Tools for the Measurement of Posttraumatic Stress Disorder Among US Military Veterans

Lisa A Brenner et al. JAMA Netw Open. .

Abstract

Importance: Veterans from recent and past conflicts have high rates of posttraumatic stress disorder (PTSD). Adaptive testing strategies can increase accuracy of diagnostic screening and symptom severity measurement while decreasing patient and clinician burden.

Objective: To develop and validate a computerized adaptive diagnostic (CAD) screener and computerized adaptive test (CAT) for PTSD symptom severity.

Design, setting, and participants: A diagnostic study of measure development and validation was conducted at a Veterans Health Administration facility. A total of 713 US military veterans were included. The study was conducted from April 25, 2017, to November 10, 2019.

Main outcomes and measures: The participants completed a PTSD-symptom questionnaire from the item bank and provided responses on the PTSD Checklist for Diagnostic and Statistical Manual of Mental Disorders, Fifth Edition (DSM-5) (PCL-5). A subsample of 304 participants were interviewed using the Clinician-Administered Scale for PTSD for DSM-5.

Results: Of the 713 participants, 585 were men; mean (SD) age was 52.8 (15.0) years. The CAD-PTSD reproduced the Clinician-Administered Scale for PTSD for DSM-5 PTSD diagnosis with high sensitivity and specificity as evidenced by an area under the curve of 0.91 (95% CI, 0.87-0.95). The CAT-PTSD demonstrated convergent validity with the PCL-5 (r = 0.88) and also tracked PTSD diagnosis (area under the curve = 0.85; 95% CI, 0.79-0.89). The CAT-PTSD reproduced the final 203-item bank score with a correlation of r = 0.95 with a mean of only 10 adaptively administered items, a 95% reduction in patient burden.

Conclusions and relevance: Using a maximum of only 6 items, the CAD-PTSD developed in this study was shown to have excellent diagnostic screening accuracy. Similarly, using a mean of 10 items, the CAT-PTSD provided valid severity ratings with excellent convergent validity with an extant scale containing twice the number of items. The 10-item CAT-PTSD also outperformed the 20-item PCL-5 in terms of diagnostic accuracy. The results suggest that scalable, valid, and rapid PTSD diagnostic screening and severity measurement are possible.

PubMed Disclaimer

Conflict of interest statement

Conflict of Interest Disclosures: Dr Brenner reported receiving fees as a consultant for sports leagues and has received royalties from American Psychological Association Publishing. Dr Gibbons has been an expert witness for the US Department of Justice, Merck, GlaxoSmithKline, Pfizer, and Wyeth and is a founder of Adaptive Testing Technologies, which distributes the Computerized Adaptive Test–Posttraumatic Stress Disorder–Mental Health (CAT-MH) battery of adaptive tests in which CAT–Posttraumatic Stress Disorder (PTSD) is included. The terms of this arrangement have been reviewed and approved by the University of Chicago in accordance with its conflict of interest policies. Dr Frank is a founder of Adaptive Testing Technologies, which distributes the CAT-MH battery of adaptive tests. She has an equity interest in Adaptive Testing Technologies Inc and in HealthRhythms Inc and is a founder and an employee of HealthRhythms Inc. She has received royalties from Guilford Press and the American Psychological Association Press. Dr Kupfer is a founder of Adaptive Testing Technologies, which distributes the CAT-MH battery of adaptive tests. He has an equity interest in Adaptive Testing Technologies Inc and in HealthRhythms Inc, of which he is a founder. He is a board member and holds an equity interest in Minerva Neuroscience and has received royalties for the Pittsburgh Sleep Quality Index from the University of Pittsburgh. Dr Gibbons is a founder of Adaptive Testing Technologies Inc, which distributes the CAT-MH battery of adaptive tests. The terms of this arrangement have been reviewed and approved by the University of Chicago in accordance with its conflict of interest policies. He has served as an expert witness regarding statistical issues for drug safety for Merck, Pfizer, GlaxoSmithKline, and the US Department of Justice.

Figures

Figure.
Figure.. Receiver Operating Characteristic Curve for the Computerized Adaptive Diagnostic (CAD) Posttraumatic Stress Disorder (PTSD) Measure
With the receiver operating characteristic curve for the prediction of the Diagnostic and Statistical Manual of Mental Disorders, Fifth Edition diagnosis of PTSD based on the CAD-PTSD classifier, the area under the curve was 0.91. FPR indicates false-positive rate; TPR, true-positive rate.
See this image and copyright information in PMC

References

    1. Fulton JJ, Calhoun PS, Wagner HR, et al. . The prevalence of posttraumatic stress disorder in Operation Enduring Freedom/Operation Iraqi Freedom (OEF/OIF) veterans: a meta-analysis. J Anxiety Disord. 2015;31(31):98-107. doi:10.1016/j.janxdis.2015.02.003 - DOI - PubMed
    1. Magruder KM, Frueh BC, Knapp RG, et al. . Prevalence of posttraumatic stress disorder in Veterans Affairs primary care clinics. Gen Hosp Psychiatry. 2005;27(3):169-179. doi:10.1016/j.genhosppsych.2004.11.001 - DOI - PubMed
    1. Gradus JL. Epidemiology of PTSD. US Department of Veterans Affairs. Accessed June 23, 2020. https://www.ptsd.va.gov/professional/treat/essentials/epidemiology.asp
    1. American Psychiatric Association . What is posttraumatic stress disorder? Accessed March 8, 2020. https://www.psychiatry.org/patients-families/ptsd/what-is-ptsd
    1. Gibbons RD, deGruy FV. Without wasting a word: extreme improvements in efficiency and accuracy using computerized adaptive testing for mental health disorders (CAT-MH). Curr Psychiatry Rep. 2019;21(8):67. doi:10.1007/s11920-019-1053-9 - DOI - PubMed

Publication types