Explaining differential item functioning focusing on the crucial role of external information - an example from the measurement of adolescent mental health

Abstract

Background: An overarching objective in research comparing different sample groups is to ensure that the reported differences in outcomes are not affected by differences between groups in the functioning of the measurement instruments, i.e. the items have to work in the same way for the different sample groups to be compared. Lack of invariance across sample groups are commonly called Differential Item Functioning (DIF). There is a sense in which the DIF of an item can be taken account of by resolving (splitting) the item into group specific items, rather than deleting the item. Resolving improves fit, retains the reliability and content provided by the item, and compensates for the DIF in estimation of person parameters on the scale of the instrument. However, it destroys invariance of the item's parameter value among the groups. Whether or not a DIF item should be resolved depends on whether the source of the DIF is relevant or irrelevant for the content of the variable. The present paper shows how external information can be used to investigate if the gender DIF found in the item "Stomach ache" in a psychosomatic symptoms scale used among adolescents may reflect abdominal pain because of a biological factor, the girls' menstrual periods.

Methods: Swedish data from the international Health Behaviour in School-aged Children study (HBSC) collected in 2005/06, 2009/10 and 2013/14 were used, comprising a total of 18,983 students in grades 5, 7 and 9. A composite measure of eight items of psychosomatic problems was analysed for DIF with respect to gender and menstrual periods using the Rasch model.

Results: The results support the hypothesis that the source of the gender DIF for the item "Stomach ache" is a gender specific biological factor. In that case the DIF should be resolved if the psychosomatic measure is not intended to tap information about abdominal pain caused by a gender specific biological factor. In contrast, if the measure is intended to tap such information, the DIF should not be resolved.

Conclusions: The conceptualisation of the measure governs whether the item showing DIF should be resolved or not.

Keywords: Differential item functioning (DIF); Psychosomatic problems; Rasch model; Resolving for DIF; Source of DIF; Validity and reliability, HBSC.

Fig. 1

Item Stomach ache showing period DIF, in a set where item Felt low is resolved for period-DIF

Fig. 2

Item Stomach ache resolved for period-DIF, in a set where first item Felt low is resolved for period DIF