Skip to main page content
U.S. flag

An official website of the United States government

Dot gov

The .gov means it’s official.
Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you’re on a federal government site.

Https

The site is secure.
The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

Access keys NCBI Homepage MyNCBI Homepage Main Content Main Navigation
. 2023 Apr;55(3):1226-1243.
doi: 10.3758/s13428-022-01867-8. Epub 2022 May 26.

The effects of survey mode on self-reported psychological functioning: Measurement invariance and latent mean comparison across face-to-face and web modes

Gaja Zager Kocjan  1 , Darja Lavtar  2 , Gregor Sočan  3
Affiliations

The effects of survey mode on self-reported psychological functioning: Measurement invariance and latent mean comparison across face-to-face and web modes

Gaja Zager Kocjan et al. Behav Res Methods. 2023 Apr.

Abstract

Despite the increasing popularity of mixed-mode approaches to data collection, studies examining measurement equivalence across different survey modes in representative samples of the general population, particularly focusing on measures of socially sensitive psychological constructs, are sparse. In this study, we used data from a large representative sample of the Slovenian population (N = 9,900) collected as part of the third wave of the European Health Interview Survey (EHIS) to examine mode-inherent effects (i.e., measurement effects that cannot be neutralized by clever survey design) of the traditional interviewer-mediated face-to-face mode and the increasingly popular self-administered web mode on three measures of psychological functioning, namely the Patient Health Questionnaire Depression Scale (PHQ-8), the Satisfaction with Life Scale (SWLS), and the Mental Health Continuum - Short Form (MHC-SF). After controlling for self-selection propensities, our results showed strict measurement invariance for all three scales across the two survey modes, but statistically significant and notable differences in latent means, suggesting that individuals who respond face-to-face systematically report better psychological functioning than individuals who respond over the web. These results suggest significant mode inherent effects that cannot be attributed to measurement non-invariance of the scales between face-to-face and web survey modes, but most likely to social desirability bias in responses achieved in the presence of an interviewer. Administration mode effects must be considered when interpreting and comparing results obtained through different survey modes, particularly interviewer-mediated versus self-administered modes, especially when using measures of culturally desirable traits and behaviors, such as mental health and well-being.

Keywords: Administration mode effects; Measurement invariance; Psychological functioning; Social desirability bias.

PubMed Disclaimer

References

    1. Akaike, H. (1973). Information theory and an extension of the maximum likelihood principle. In Petrov, B. N., & Csaki, F. (Eds.), 2nd International Symposium on Information Theory (pp. 267–281.). Akademiai Kiado.
    1. American Psychiatric Association. (2013). Diagnostic and statistical manual of mental disorders (5th ed.). American Psychiatric Publishing.
    1. Baas, K. D., Cramer, A. O., Koeter, M. W., Van De Lisdonk, E. H., Van Weert, H. C., & Schene, A. H. (2011). Measurement invariance with respect to ethnicity of the Patient Health Questionnaire–9 (PHQ–9). Journal of Affective Disorders, 129(1–3), 229–235. https://doi.org/10.1016/j.jad.2010.08.026 - DOI - PubMed
    1. Bäckström, M., & Björklund, F. (2014). Social desirability in personality inventories: The nature of the evaluative factor. Journal of Individual Differences, 35(3), 144–157. https://doi.org/10.1027/1614-0001/a000138 - DOI
    1. Bäckström, M., Björklund, F., & Larsson, M. R. (2012). Social desirability in personality assessment: Outline of a model to explain individual differences. In M. Ziegler, C. MacCann, & R. D. Roberts (Eds.), New perspectives on faking in personality assessment (pp. 201–213). Oxford University Press.

Publication types

LinkOut - more resources