Item performance of the scale for the assessment and rating of ataxia in rare and ultra-rare genetic ataxias

Abstract

The Scale for the Assessment and Rating of Ataxia (SARA) is widely used for assessing the severity and progression of genetic cerebellar ataxias. SARA is now considered a primary end point in several ataxia treatment trials, but its underlying composite item measurement model has not yet been tested. This work aimed to evaluate the composite properties of SARA and its items using item response theory (IRT) and to demonstrate its applicability across even ultra-rare genetic ataxias. Leveraging SARA subscores data from 1932 visits from 990 patients of the Autosomal Recessive Cerebellar Ataxias (ARCA) registry, we assessed the performance of SARA using IRT methodology. The item characteristics were evaluated over the ataxia severity range of the entire ataxia population as well as the assessment validity across 115 genetic ARCA subpopulations. A unidimensional IRT model was able to describe SARA item data, indicating that SARA captures one single latent variable. All items had high discrimination values (1.5-2.9) indicating the effectiveness of the SARA in differentiating between subjects with different disease statuses. Each item contributed between 7% and 28% of the total assessment informativeness. There was no evidence for differences between the 115 genetic ARCA subpopulations in SARA applicability. These results show the good discrimination ability of SARA with all of its items adding informational value. The IRT framework provides a thorough description of SARA on the item level, and facilitates its utilization as a clinical outcome assessment in upcoming longitudinal natural history or treatment trials, across a large number of ataxias, including ultra-rare ones.

FIGURE 1

Item characteristic curves for the different items of the SARA scale, describing the probability of occurrence of each score Y for a patient with a given ataxia severity level. Ataxia severity represents a Z‐score relative to the typical individual in the reference population, that is, the entire ARCA cohort.

FIGURE 2

Data correlations between different item pairs (referred to as the first and second item) of the SARA scale for both original ARCA dataset and 100 simulated datasets. Upper matrix: item‐pairs correlations for the observed dataset (with value‐based color scaling) and item‐pairs average correlations for the simulated datasets. Lower matrix: The black lines represent the median (solid lines) and the 5th and 95th percentiles (dashed lines) of the observed item scores (blue circles). The shaded areas represent the 95% confidence intervals of the corresponding percentiles for the simulated data (blue: upper and lower percentiles, pink: median percentile).

FIGURE 3

Residual correlations between different item pairs (referred to as the 1st and 2nd item) of the SARA scale for both original ARCA dataset and 100 simulated datasets. Upper matrix: item‐pairs residual correlations for the observed dataset (with value‐based color scaling) and item‐pairs average residual correlations for the simulated datasets. Lower matrix: The black lines represent the median (solid lines) and the 5th and 95th percentiles (dashed lines) of the observed item scores. The shaded areas represent the 95% confidence intervals of the corresponding percentiles for the simulated data (blue: upper and lower, pink: median).

FIGURE 4

Item information curves for the different items in the SARA scale as a function of ataxia disease severity ($\psi$). The shaded areas indicate the ataxia severity interval for 95% of the studied population. The reported I _j is the total population information for each item; that is, the integration of item information over the full $\psi$ range.

FIGURE 5

Difference in means of iOFVs (red points); $\text{mean}\left({\text{iOFV}}_{\text{subpop}}\right)-\text{mean}\left({\text{iOFV}}_{\mathrm{all}\mathrm{pop}}\right)$, between different genetic ARCA subpopulations and the entire ARCA population, along with the 95% confidence intervals (error bars). n is the number of subjects in each subpopulation group. One‐subject diagnoses is a pool of diagnoses comprising only one subject. Genetically undetermined: the pool of patients with genetically yet unidentified diagnoses, despite extensive genetic work‐up. Permuted group is the hypothetical population where the item subscores of the original dataset were permuted among individuals. The results of subpopulations with n ≤ 2 are not illustrated in the figure.