Determining the Generalizability of the PROMIS Depression Domain's Floor Effect and Completion Time in Patients Undergoing Orthopaedic Surgery

Abstract

Background: Depression is common and can decrease the likelihood that a patient undergoing an orthopaedic procedure will improve after that intervention. Research at a single institution using the Patient-reported Outcomes Measurement Information System (PROMIS) showed that the PROMIS Depression domain had a substantial floor effect (lowest depressive symptoms measurable) in patients with orthopaedic conditions, although the instrument is designed to follow a population-based normal distribution. Patients with the lowest scores (floor) completed the questionnaire more quickly than those who did not have the lowest scores, which suggests that patients may not want to report or discuss depressive symptoms with their orthopaedic surgeon.

Questions/purposes: (1) Do the scores of a large population of patients with orthopaedic conditions completing the PROMIS Physical Function (PF), Pain Interference (PI), and Depression questionnaires follow the normal population distribution used to calibrate the PROMIS instrument? (2) Do the scores of a large population of patients with foot and ankle, spine, upper extremity, multiple orthopaedic subspecialty, or other orthopaedic conditions completing the PROMIS Depression questionnaire follow the normal population distribution used to calibrate the PROMIS instrument? (3) Do all orthopaedic patients with the lowest possible score (floor effect) on the PROMIS Depression questionnaire answer questions more quickly than those who do not?

Methods: Between February 12, 2015 and July 19, 2018, a total of 513,042 patients with orthopaedic conditions at a single urban, tertiary academic medical center were asked to complete the PROMIS PF, PI, and Depression domains. The PROMIS PF, PI, and Depression questionnaires were administered at 433,939 patient encounters (85%), and 325,251 of 433,939 (75%) completed all three questionnaires and were included in our final sample. The remaining encounters were excluded from all analyses because of incomplete PROMIS data. Using this large sample size, we determined the distributions of the PROMIS PF, PI, and Depression domains for all patients with orthopaedic conditions by visual inspection. We calculated the percentage of patients with the lowest possible score on the PROMIS Depression domain ("floor group") for all patients with orthopaedic conditions, as well for patients presenting to orthopaedic subspecialists (foot and ankle, spine, upper extremity, multiple subspecialties, or other specialists). We compared patient and PROMIS characteristics between patients in the floor group and all others ("standard group") using a bivariate analysis, including the total time it took to complete the questionnaires and average time it took to answer each question.

Results: Of the three PROMIS domains analyzed, only the PROMIS PF demonstrated a normal distribution on visual inspection. PROMIS PI scores had a measurable ceiling effect (more pain symptoms) (15,520 of 325,251 patient encounters; 5%), while PROMIS Depression scores demonstrated a clear floor effect (65,226 of 325,251; 20%). When analyzed by subspecialty, there was a clear floor effect for the PROMIS Depression in the specialties of foot and ankle (6031 of 29,273 patient encounters; 21%), spine (7611 of 58,266; 13%), upper extremity (9851 of 42,864; 23%), multiple subspecialties (276 of 1662; 17%), and other (41, 454 of 193,185; 22%). Generally, those in the floor group spent less time per question on the PROMIS Depression questionnaire than did those in the standard group (5.3 seconds ± 2.6 seconds versus 8.1 seconds ± 4.6 seconds; mean difference, 2.8 seconds; p < 0.001).

Conclusions: The substantial floor effect of the PROMIS Depression in the setting of hasty completion raises the question of whether such results truly indicate that these patients have minimal depression or whether they simply do not report their true mental health status in an orthopaedic setting. When such patients are seen in clinic, surgeons may benefit from using the PROMIS Depression as an educational opportunity to explain the importance of mental health in orthopaedic care and ensure the questionnaire is completed accurately. This study builds on previous work by suggesting these findings apply to other general and subspecialty orthopaedic patient populations at academic medical centers. Future work may seek to determine the best way of ensuring that the PROMIS Depression questionnaire is completed accurately in orthopaedic clinics.

Level of evidence: Level II, diagnostic study.

Fig. 1

This STROBE diagram illustrates the pathway from the total possible patient sample to the final number used for all analyses.

Fig. 2

This figure shows (A) a Q-Q plot for the PROMIS PF, which demonstrates a normal distribution; (B) a Q-Q plot for the PROMIS PI, which has a ceiling effect; and (C) a Q-Q plot for the PROMIS Depression, which has a floor effect.