Assessing differential item functioning for the Social Appearance Anxiety Scale: a Scleroderma atient-centred Intervention Network (SPIN) Cohort Study

Abstract

Objectives: The Social Appearance Anxiety Scale (SAAS) is a 16-item questionnaire developed to evaluate fear of appearance-based evaluation by others. The primary objective of this research was to investigate the existence of differential item functioning (DIF) for the 16 SAAS items, comparing patients who completed the SAAS in English and French, either to confirm that scores are comparable or provide guidance on calculating comparable scores. A secondary research objective was to investigate the existence of DIF based on sex and disease status. A tertiary research objective was to assess DIF related to language, sex, and disease status on the recently developed SAAS-5.

Design: This was a cross-sectional analysis using baseline data from patients enrolled in the Scleroderma Patient-centred Intervention Network (SPIN).

Setting: SPIN patients included in the present study were enrolled at 43 centres in Canada, USA, UK, France and Australia, with questionnaires completed in April 2014 to July 2019.

Participants: 1640 SPIN patients completed the SAAS in French (n=600) or English (n=1040).

Primary and secondary measures: The SAAS was collected along with demographic and disease characteristics.

Results: Six items were identified with statistically significant language-based DIF, four with sex-based DIF and one with disease type-based DIF. However, factor scores before and after accounting for DIF were similar (Pearson correlation >0.99), and individual score differences were small. This was true for both the full and shortened versions of the SAAS.

Conclusion: SAAS and SAAS-5 scores are comparable across language, sex, and disease-type, despite small differences in how patients respond to some items.

Keywords: anxiety disorders; rheumatology; statistics & research methods.

Figure 1

Three possible ICC for a five-category item. The left and middle panels show ICCs for items with the same approximate discrimination parameters (alphas) but different item-level thresholds (betas). The left and right panels show ICCs for items with the same approximate item-level thresholds (betas) but different discrimination parameters (alphas). ICC, item-characteristic curve.

Figure 2

Item true score functions for the six items identified as having language-based DIF. For items 5, 8, 12 and 13, these plots demonstrate that French speakers are expected to give larger categorical responses than English speakers with equal levels of appearance anxiety. This trend is reversed for item 2, while item 11 demonstrates non-uniform DIF (ie, the true score functions for English and French speakers cross each other). DIF, differential item functioning.

Figure 3

Test characteristic curve showing expected summed scores on the SAAS as a function of estimated social appearance anxiety accounting for DIF. Thus, among French and English speakers with the same estimated level of social appearance anxiety, French speakers are expected to have slightly larger summed scores. DIF, differential item functioning; SAAS, Social Appearance Anxiety Scale.

Figure 4

The top plot shows GPCM score differences at the questionnaire level (accounting for DIF—not accounting for DIF) compared with factor scores accounting for DIF. The largest score differences occur at estimated appearance anxiety levels.5 SD below average and 1 SD above average. The figure on the bottom left shows a box plot of these score differences among all respondents. The figure on the bottom right shows these differences by language. Overall differences are small and are mostly negative for English speakers and positive for French speakers, suggesting that pooled scores from a GPCM will tend to overestimate appearance anxiety for French speakers and underestimate it for English speakers. DIF, differential item functioning; GPCM, generalised partial credit model.