Skip to main page content
U.S. flag

An official website of the United States government

Dot gov

The .gov means it’s official.
Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you’re on a federal government site.

Https

The site is secure.
The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

Access keys NCBI Homepage MyNCBI Homepage Main Content Main Navigation
. 2022 May 1;13(4):621-646.
doi: 10.1332/175795921X16494126913909.

Data quality and response distributions in a mixed-mode survey

Mary Beth Ofstedal  1 , Gábor Kézdi  1 , Mick P Couper  1
Affiliations

Data quality and response distributions in a mixed-mode survey

Mary Beth Ofstedal et al. Longit Life Course Stud. .

Abstract

Longitudinal surveys traditionally conducted by interviewers are facing increasing pressures to explore alternatives such as sequential mixed-mode designs, which start with a cheaper self-administered mode (online) then follow up using more expensive methods such as telephone or face-to-face interviewing. Using a designed experiment conducted as part of the 2018 wave of the Health and Retirement Study (HRS) in the US, we compare a sequential mixed-mode design (web then telephone) with the standard telephone-only protocol. Using an intent-to-treat analysis, we focus on response quality and response distributions for several domains key to HRS: physical and psychological health, financial status, expectations and family composition. Respondents assigned to the sequential mixed-mode (web) had slightly higher missing data rates and more focal responses than those assigned to telephone-only. However, we find no evidence of differential quality in verifying and updating roster information. We find slightly lower rates of asset ownership reported by those assigned to the web mode. Conditional on ownership, we find no detectable mode effects on the value of assets. We find more negative (pessimistic) expectations for those assigned to the web mode. We find little evidence of poorer health reported by those assigned to the web mode. We find that effects of mode assignment on measurement are present, but for most indicators the effects are small. Finding ways to remediate the differences in item-missing data and focal values should help reduce mode effects in mixed-mode surveys or those transitioning from interviewer- to self-administration.

Keywords: intent-to-treat analysis; mode effects; sequential mixed-mode; telephone; web.

PubMed Disclaimer

Conflict of interest statement

Conflict of interest

The authors declare that there is no conflict of interest.

Figures

Figure 1.
Figure 1.
Measures of data quality and response distributions, by survey domain
See this image and copyright information in PMC

References

    1. Al Baghal T and Lynn P (2015) Using motivational statements in web-instrument design to reduce item-missing rates in a mixed-mode context, Public Opinion Quarterly, 79(2): 568–579.
    1. Bianchi A, Biffignandi S and Lynn P (2017) Web-face-to-face mixed mode design in a longitudinal survey: Effects on participation rates, sample composition, and costs, Journal of Official Statistics, 33(2): 385–408.
    1. Bowyer BT and Rogowski JC (2017) Mode matters: Evaluating response comparability in a mixed-mode survey, Political Science Research and Methods, 5(2): 295–313.
    1. Bruine de Bruin W, van der Klaauw W, van Rooij M, Teppa F and de Vos K (2016) Measuring expectations of inflation: Effects of survey mode, wording, and opportunities to revise. DNB working paper no. 506, Amsterdam: De Nederlandsche Bank. .
    1. Burton J, and Jäckle A (2020) Mode effects, Understanding Society working paper series No. 2020–05, Colchester: University of Essex, ISER.

Publication types

LinkOut - more resources