Mail surveys: obsolescent model or valuable instrument in general practice research?

Abstract

Question under study: Due to low response rates mail surveys have been called into question as research instruments in general practice. The most effective actions to reduce non-response, such as financial incentives and complex follow-up procedures, are costly. We investigated whether a good response rate is achievable with a less costly survey design, and examined the effect of increased response rates due to repeated follow-ups on survey results.

Methods: In a mail survey on drug interactions among 2000 general practitioners in south-west Germany, most well-known criteria influencing response rates were met except financial incentives. A four stage design with two reminders was applied and the time course of response was recorded. Results after both reminders were calculated with 95% confidence intervals and compared with initial results using the Jonckheere-Terpstra test with correction for multiple testing. A p <0.01 was considered significant.

Results: Although we did not provide financial incentives we achieved a response rate of 60.8% with our survey design. The first reminder with a simple postcard was almost three times less effective than the second reminder including another copy of the questionnaire. For only two survey questions, the answers of late respondents differed significantly from those of initial respondents (p <0.01). For these two questions, cumulative results after both reminders never differed from initial results by more than 3.7%.

Conclusion: Even if financial incentives are not affordable, good response rates can be obtained among general practitioners when surveys are meticulously planned and implemented. Potential non-response bias introduced by those general practitioners who do not answer despite numerous reminders, cannot be tested by comparing early and late respondents. Therefore, we suggest that the impact of reminders on survey results should be assessed early. If no bias can be detected one further reminder with a copy of the questionnaire might result in estimates very similar to those after numerous reminders.