Improving the Delivery of Function-Directed Care During Acute Hospitalizations: Methods to Develop and Validate the Functional Assessment in Acute Care Multidimensional Computerized Adaptive Test (FAMCAT)

Abstract

Objective: To (1) develop a patient-reported, multidomain functional assessment tool focused on medically ill patients in acute care settings; (2) characterize the measure's psychometric performance; and (3) establish clinically actionable score strata that link to easily implemented mobility preservation plans.

Design: This article describes the approach that our team pursued to develop and characterize this tool, the Functional Assessment in Acute Care Multidimensional Computer Adaptive Test (FAMCAT). Development involved a multistep process that included (1) expanding and refining existing item banks to optimize their salience for hospitalized patients; (2) administering candidate items to a calibration cohort; (3) estimating multidimensional item response theory models; (4) calibrating the item banks; (5) evaluating potential multidimensional computerized adaptive testing (MCAT) enhancements; (6) parameterizing the MCAT; (7) administering it to patients in a validation cohort; and (8) estimating its predictive and psychometric characteristics.

Setting: A large (2000-bed) Midwestern Medical Center.

Participants: The overall sample included 4495 adults (2341 in a calibration cohort, 2154 in a validation cohort) who were admitted either to medical services with at least 1 chronic condition or to surgical/medical services if they required readmission after a hospitalization for surgery (N=4495).

Intervention: Not applicable.

Main outcome measures: Not applicable.

Results: The FAMCAT is an instrument designed to permit the efficient, precise, low-burden, multidomain functional assessment of hospitalized patients. We tried to optimize the FAMCAT's efficiency and precision, as well as its ability to perform multiple assessments during a hospital stay, by applying cutting edge methods such as the adaptive measure of change (AMC), differential item functioning computerized adaptive testing, and integration of collateral test-taking information, particularly item response times. Evaluation of these candidate methods suggested that all may enhance MCAT performance, but none were integrated into initial MCAT parameterization.

Conclusions: The FAMCAT has the potential to address a longstanding need for structured, frequent, and accurate functional assessment among patients hospitalized with medical diagnoses and complications of surgery.

Keywords: AM-PAC, Activity Measure of Post-Acute Care; AMC, Adaptive Measurement of Change; Activities of daily living; CAT, computerized adaptive testing; Cognition; DIF, differential item functioning; EHR, electronic health record; FAM, Functional Assessment for Acute Care Multidimensional; FAMCAT, Functional Assessment in Acute Care Multidimensional Computer Adaptive Test; HIPAA, Health Insurance Portability and Accountability Act of 1996; IRT, item response theory; MCAT, multidimensional computerized adaptive testing; MGRM, multidimensional graded response model; MIRT, multidimensional item response theory; PAC, postacute care; PH, physical function; PROM, patient-reported outcome measure; PROMIS, Patient-Reported Outcomes Measurement Information System; Rehabilitation; SF, short form.

Fig 1

Anticipated integration of FAMCAT testing during and following a typical hospital stay.

Fig 2

Sequential steps in FAMCAT development and testing.

Fig 3

Participant flow diagram for calibration and validation cohorts. *An initial batch 1 data export was performed after 500 participants had been assessed to identify linking items. The identification of linking items prior to completing batch 1 data collection allowed a seamless transition from batch 1 to batch 2 collection because batch 2 included the linking items. Data from this initial pull were used for the MIRT models. ^†Responses were retained from calibration cohort members who answered at least 90% of the administered items. ^‡The complete calibration cohort data set was used for the DIF analyses. These data differed in that they included the batch 1 data collected following the initial export.

Fig 4

Four hypothesized levels for each FAMCAT domain that inform individualized mobility preservation plans.

Fig 5

Smoothed frequency distributions of the basic mobility MIRT model estimates for subgroups classified by nurse mobility ratings.