Skip to main page content
U.S. flag

An official website of the United States government

Dot gov

The .gov means it’s official.
Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you’re on a federal government site.

Https

The site is secure.
The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

Access keys NCBI Homepage MyNCBI Homepage Main Content Main Navigation
. 2018 Mar;6(1):72-97.
doi: 10.1093/jssam/smx011. Epub 2017 Jul 6.

A Sequential Mixed-Mode Experiment in the U.S. National Monitoring the Future Study

Megan E Patrick, Mick P Couper, Virginia B Laetz, John E Schulenberg, Patrick M O'Malley, Lloyd D Johnston, Richard A Miech

A Sequential Mixed-Mode Experiment in the U.S. National Monitoring the Future Study

Megan E Patrick et al. J Surv Stat Methodol. 2018 Mar.

Abstract

The national Monitoring the Future (MTF) study examines substance use among adolescents and adults in the United States and has used paper questionnaires since it began in 1975. The current experiment tested three conditions as compared to the standard MTF follow-up protocol (i.e., MTF Control) for the first MTF follow-up survey at ages 19/20 years (i.e., one or two years after high school graduation). The MTF Control group included participants who completed in-school baseline surveys in the 12th grade in 2012-2013 and who were selected to participate in the first follow-up survey in 2014 (n = 2,451). A supplementary sample of participants who completed the 12th grade baseline survey in 2012 or 2013 but were not selected to participate in the main MTF follow-up (n = 4,950) were recruited and randomly assigned to one of three experimental conditions: (1) Mail Push, (2) Web Push, (3) Web Push + E-mail. Results indicated that the overall response rate was lower in Condition 2 compared to MTF Control and to Condition 1; there were no differences between Condition 3 and other conditions. Web response was highest in Condition 3; among web responders, smartphone response was also highest in Condition 3. Subgroup differences also emerged such that, for example, compared to white participants, Hispanics had greater odds of web (versus paper) response and blacks had greater odds of smartphone (versus computer or tablet) response. Item nonresponse was lowest in the Web Push conditions (compared to MTF Control) and on the web survey (compared to paper). Compared to MTF Control, Condition 3 respondents reported higher rates of alcohol use in the past 30 days. The total cost was lowest for Condition 3. Overall, the Condition 3 Web Push + E-mail design is promising. Future research is needed to continue to examine the implications of web and mobile response in large, national surveys.

PubMed Disclaimer

Figures

Figure 1.
Figure 1.
Overall Response Rates by Condition. *Response rates differ at p < 0.05. Number of responders by condition: Control Group n = 879, Condition 1 n = 634, Condition 2 n = 566, Condition 3 n = 610.
See this image and copyright information in PMC

References

    1. Bachman J. G., Johnston L. D., O'Malley P. M., Schulenberg J. E., Miech R. A. (2015), The Monitoring the Future Project after Four Decades: Design and Procedures (Monitoring the Future Occasional Paper No. 82), Ann Arbor, MI: Institute for Social Research.
    1. Bandilla W., Couper M. P., Kaczmirek L. (2012), “ The Mode of Invitation for Web Surveys,” Survey Practice, 53 Available at: <http://www.surveypractice.org/index.php/SurveyPractice/article/view/20>. Date accessed: 15 January. 2017.
    1. Bandilla W., Couper M. P., Kaczmirek L. (2014), “ The Effectiveness of Mailed Invitations for Web Surveys and the Representativeness of Mixed-Mode Versus Internet-Only Samples,” Survey Practice, 74 Available at: <http://www.surveypractice.org/index.php/SurveyPractice/article/view/274>. Date accessed: 15 January. 2017.
    1. Bensky E. N., Link M., Shuttles C. (2010), “ Does the Timing of Offering Multiple Modes of Return Hurt the Response Rates?” Survey Practice, 35 Available at: <http://www.surveypractice.org/index.php/SurveyPractice/article/view/146>. Date accessed: 15 January. 2017.
    1. Biemer P. P., Murphy J., Zimmer S., Berry C., Deng G., Lewis K. (2016), “A Test of Web/Papi Protocols and Incentives for the Residential Energy Consumption Survey,” paper presented at the Annual Meeting of the American Association for Public Opinion Research, Austin, TX.