A comparison of mail and telephone administration of district-level questionnaires for the School Health Policies and Programs Study (SHPPS) 2006: effects on estimates and data quality

Abstract

Background: The School Health Policies and Programs Study (SHPPS) is a national study periodically conducted to assess school health policies and programs at the state, district, school, and classroom levels. For SHPPS 2006, district-level questionnaires were designed for telephone administration, but mixed-mode data collection that also used paper-and-pencil mail questionnaires was required to obtain an acceptable response rate. Because most mode effect research has involved person-level rather than institution-level data, little is known about the effects of mixed-mode data collection on data quality and prevalence estimates obtained through surveys of school personnel.

Methods: SHPPS 2006 used 1-stage stratified cluster sampling to select a nationally representative sample of public school districts. Personnel in about half of the 538 responding districts completed paper questionnaires and returned them via mail. Analyses were performed comparing data quality and prevalence estimates for mail and telephone administration.

Results: Prevalence estimates for only 7.0% (39) of 554 questions tested across the 7 questionnaires differed significantly by response mode at the p < .01 level. Regarding data quality, use of the "don't know" response was higher for telephone administration.

Conclusions: The results of this study demonstrate that SHPPS 2006 successfully used a mixed-mode approach, allowing the data to be used without concern about the mixed-mode administration. The results may also be useful to other researchers interested in using surveys to collect data on schools or school districts or other data that is not person level.