Inaccurate Use of the Upper Extremity Fugl-Meyer Negatively Affects Upper Extremity Rehabilitation Trial Design: Findings From the ICARE Randomized Controlled Trial

Abstract

Objective: To determine the extent to which estimates of sample and effect size in stroke rehabilitation trials can be affected by simple summation of ordinal Upper Extremity Fugl-Meyer (UEFM) items compared with a Rasch-rescaled UEFM.

Design: Rasch analysis of Interdisciplinary Comprehensive Arm Rehabilitation Evaluation (ICARE) phase III trial data, comparing 3 upper extremity (UE) motor treatments in stroke survivors enrolled 45.8±22.4 days poststroke. Participants underwent a structured UE motor training known as the Accelerated Skill Acquisition Program, usual and customary care, or dose-equivalent care. UEFM data from baseline, postintervention, and 6 and 12 months later were included for analysis.

Setting: Outpatient stroke rehabilitation.

Participants: ICARE participants (N=361).

Interventions: Not applicable.

Main outcome measures: Item difficulties, person abilities, and sample size.

Results: Because of their ordinality, summed raw UEFM scores measured motor impairment inconsistently across different ranges of stroke severity relative to the rescaled UEFM. In the full ICARE sample, raw UEFM understated scores relative to the rescaled UEFM by 7.4 points for the most severely impaired, but overstated scores by up to 8.4 points toward the ceiling. As a result, 50.9% of all UEFM observations showed a residual error greater than 10% of the total UEFM score. Relative to the raw scores, the rescaled UEFM improved the effect size of change in motor impairment between baseline and 1 year (d=0.35). For a hypothetical 3-arm trial resembling ICARE, UEFM rescaling reduced the required sample size by 32% (n=108) compared with raw UEFM (n=159).

Conclusions: In UE rehabilitation trials, a rescaled UEFM potentially decreases sample size by one-third, decreasing costs, duration, and the number of subjects exposed to experimental risks. This benefit is obtained through increased measurement efficiency. Reductions in ceiling effects are also possible. These findings apply to ICARE-like trials. Confirmatory validation in another phase III trial is needed.

Keywords: Clinical trials as topic; Data accuracy; Disability evaluation; Recovery of function; Rehabilitation; Sample size; Stroke; Stroke rehabilitation; Upper extremity.

Figure 1.

Raw UEFM score measures impairment inconsistently over stroke severity. Graph shows raw UEFM scores (x axis) vs. the difference between raw and rescaled UEFM (y axis) for 361 stroke survivors in the ICARE cohort. Horizontal line in gray denotes a cutoff at UEFM raw score = 27 that shows the least difference between raw and rescaled UEFM scores. This cut off also denotes the transition from under to over measurement of UE impairment by the raw UEFM scale. Individual scores improve over time during repeated measurements (baseline, post-treatment, 6-month, and 12-month), but stay consistently on the curve. E.g. white triangles (at T_1S, T_2S, T_3S, T_4S) show improved UEFM score from an exemplar ICARE participant with severely impaired UE measured at baseline, post-treatment, 6-month, and 12-month time points. Gray squares (at T_1M, T_2M, T_3M, T_4M) show improved scores from an exemplar moderately-impaired participant.

Figure 2.

Number of UEFM observations with raw-rescaled difference >MCID. A-D. Dashed vertical lines filled in light gray show ±10% window greater than minimal clinically important difference (MCID, here, 5.6). Impact of inconsistent mismeasurement as participants recover in time is seen by a bigger fraction of the sample that progressively shifts to the right, outside the 10% MCID boundaries of error over time; 32.7%, 58.5%, 55.1%, 61.1% of the sample respectively at (A) Baseline, (B) Post-treatment (4 months), (C) 6-month, and (D) 12-month. E. Impact of 1-year change in UEFM (difference between 1-year and baseline, the primary outcome), 26.3% observations fall outside the MCID window.

Figure 3.

Percentage of the sample unable to complete a task successfully (score <1.5 logits on an item, range 0–2 logits). UEFM assessment was completed at Baseline (0 months), post-treatment (4 months), 6-, and 12-months (primary outcome time point). UEFM items are arranged hierarchically with the easiest items on top. The sample improves on different UEFM items so that the proportion of items “unsuccessfully completed” shrinks between Baseline – 12-months. Most participants (75–80%) hit the ceiling on easiest items early at post-intervention (y-axis, only a small fraction of sample is “unable to complete these items successfully”), but continue to improve at different rates on the more difficult items up to 12-months.