Floor Effect of PROMIS Depression CAT Associated With Hasty Completion in Orthopaedic Surgery Patients

Abstract

Background: The Patient Reported Outcomes Measurement Information System (PROMIS) was developed to provide valid, reliable, and standardized measures to gather patient-reported outcomes for many health domains, including depression, independent of patient condition. Most studies confirming the performance of these measures were conducted with a consented, volunteer study population for testing. Using a study population that has undergone the process of informed consent may be differentiated from the validation group because they are educated specifically as to the purpose of the questions and they will not have answers recorded in their permanent health record.

Questions/purposes: (1) When given as part of routine practice to an orthopaedic population, do PROMIS Physical Function and Depression item banks produce score distributions different than those produced by the populations used to calibrate and validate the item banks? (2) Does the presence of a nonnormal distribution in the PROMIS Depression scores in a clinical population reflect a deliberately hasty answering of questions by patients? (3) Are patients who are reporting minimal depressive symptoms by scoring the minimum score on the PROMIS Depression Computer Adaptive Testing (CAT) distinct from other patients according to demographic data or their scores on other PROMIS assessments?

Methods: Univariate descriptive statistics and graphic histograms were used to describe the frequency distribution of scores for the Physical Function and Depression item banks for all orthopaedic patients 18 years or older who had an outpatient visit between June 2015 and December 2016. The study population was then broken into two groups based on whether they indicated a lack of depressive symptoms and scored the minimum score (34.2) on the Depression CAT assessment (Floor Group) or not (Standard Group). The distribution of Physical Function CAT scores was compared between the two groups. Finally, a time-per-question value was calculated for both the Physical Function and Depression CATs and was compared between assessments within each group as well as between the two groups. Bivariate statistics compared the demographic data between the two groups.

Results: Physical Function CAT scores in musculoskeletal patients were normally distributed like the distribution calibration population; however, the score distribution of the Depression CAT in musculoskeletal patients was nonnormal with a spike in the floor score. After excluding the floor spike, the distribution of the Depression CAT scores was not different from the population control group. Patients who scored the floor score on the Depression CAT took slightly less time per question for Physical Function CAT when compared with other musculoskeletal patients (floor patients: 11 ± 9 seconds; normally distributed patients: 12 ± 10 seconds; mean difference: 1 second [0.8-1.1]; p < 0.001 but not clinically relevant). They spent a substantially shorter amount of time per question on the Depression CAT (Floor Group: 4 ± 3 seconds; Standard Group: 7 ± 7 seconds; mean difference: 3 [2.9-3.2]; p < 0.001). Patients who scored the minimum score on the PROMIS Depression CAT were younger than other patients (Floor Group: 50 ± 18 SD; Standard Group: 55 ± 16 SD; mean difference: 4.5 [4.2-4.7]; p < 0.001) with a larger percentage of men (Floor Group: 48.8%; Standard Group 40.0%; odds ratio 0.6 [0.6-0.7]; p < 0.001) and minor differences in racial breakdown (Floor Group: white 85.2%, black 11.9%, other 0.03%; Standard Group: white 83.9%, black 13.7%, other 0.02%).

Conclusions: In an orthopaedic surgery population that is given PROMIS CAT as part of routine practice, the Physical Function item bank had a normal performance, but there is a group of patients who hastily complete Depression questions producing a strong floor effect and calling into question the validity of those floor scores that indicate minimal depression.

Level of evidence: Level II, diagnostic study.

Fig. 1

The graph shows a frequency distribution of scores for the PROMIS Physical Function CAT in the study population.

Fig. 2

The graph shows a frequency distribution of scores for the PROMIS Depression CAT in the study population.

Fig. 3

The graph shows a frequency distribution of scores for the PROMIS Depression CAT in Group 1.