Skip to main page content
U.S. flag

An official website of the United States government

Dot gov

The .gov means it’s official.
Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you’re on a federal government site.

Https

The site is secure.
The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

Access keys NCBI Homepage MyNCBI Homepage Main Content Main Navigation
. 2016 Dec;79(4):333-354.
doi: 10.1177/0190272516628298. Epub 2016 Nov 18.

Lies, Damned Lies, and Survey Self-Reports? Identity as a Cause of Measurement Bias

Philip S Brenner  1 , John DeLamater  2
Affiliations

Lies, Damned Lies, and Survey Self-Reports? Identity as a Cause of Measurement Bias

Philip S Brenner et al. Soc Psychol Q. 2016 Dec.

Abstract

Explanations of error in survey self-reports have focused on social desirability: that respondents answer questions about normative behavior to appear prosocial to interviewers. However, this paradigm fails to explain why bias occurs even in self-administered modes like mail and web surveys. We offer an alternative explanation rooted in identity theory that focuses on measurement directiveness as a cause of bias. After completing questions about physical exercise on a web survey, respondents completed a text message-based reporting procedure, sending updates on their major activities for five days. Random assignment was then made to one of two conditions: instructions mentioned the focus of the study, physical exercise, or not. Survey responses, text updates, and records from recreation facilities were compared. Direct measures generated bias-overreporting in survey measures and reactivity in the directive text condition-but the nondirective text condition generated unbiased measures. Findings are discussed in terms of identity.

Keywords: identity; measurement; physical exercise; survey research; time diaries.

PubMed Disclaimer

Figures

Figure 1
Figure 1
Example of Study Sequence by Reference Periods
Figure 2
Figure 2
Potential Outcomes from Comparison between Survey Response (S) and Criterion Measures (C)
Figure 3
Figure 3
Comparisons of Mean Number of Days of Exercise and Reported Exercise by Data Collection Methods and Location of Exercise Note: Tests (Wilcoxon signed-rank) use paired data; equivalence tests use TOST signed-rank. The survey and text measures are from sequential reference periods, not the identical reference period. All other comparisons are from matching reference periods. *p<.05. ***p<.001.
Figure 4
Figure 4
Comparison of Exercise Identity Prominence Ratings by Respondent Type Note: Tests use Mann-Whitney-Wilcoxon rank-sum. Equivalence tests use the TOST rank-sum. The survey and text measures are from sequential reference periods, not the identical reference period. All other comparisons are from matching reference periods. *p<.05. **p<.01. ***p<.001.
Figure 5
Figure 5
Predicted Probability of Exercising by Condition and Exercise Identity Importance *p<.05. **p<.01. ***p<.001.
See this image and copyright information in PMC

References

    1. Adams Swann Arp, Matthews Charles E, Ebbeling Cara B, Moore Charity G, Cunningham Joan E, Fulton Jeanette, Hebert James R. The Effect of Social Desirability and Social Approval on Self-Reports of Physical Activity. American Journal of Epidemiology. 2005;161:389–98. - PMC - PubMed
    1. Ainsworth Barbara E, Jacobs David R, Leon Arthur S. Validity and Reliability of Self-Reported Physical Activity Status: The Lipid Research Clinics Questionnaire. Medicine and Science in Sports and Exercise. 1992;25:92–98. - PubMed
    1. Albanes Demetrius, Conway Joan M, Taylor Philip R, Moe Paul W, Judd Joseph. Validation and Comparison of Eight Physical Activity Questionnaires. Epidemiology. 1990;1:65–71. - PubMed
    1. American Association for Public Opinion Research. Standard Definitions: Final Dispositions of Case Codes and Outcome Rates for Surveys. 8. AAPOR; 2015.
    1. Barker Kathleen, Fong Lynda, Grossman Samara, Quin Colin, Reid Rachel. Comparison of Self-Reported Recycling Attitudes and Behaviors with Actual Behavior. Psychological Reports. 1994;75:571–77.

LinkOut - more resources