Rapid surveys for program evaluation: design and implementation of an experiment in Ecuador

Abstract

This paper presents details from the field test of two rapid surveys in Ecuador in 1995. It focuses on how the surveys were designed and implemented, including descriptions of the sampling procedures, the preparation and use of preprogrammed palmtop computers for data entry, the selection criteria for the interviewing team, and how the training was designed. Lessons are drawn that will assist health professionals plan and carry out better rapid data collection in the future. The objective of the study was to evaluate the reliability and validity of data gathered during the rapid surveys as compared with a recent "gold standard" national survey. A two-way factorial design was used to control for differences in sampling (probability versus quasi-probability) and methods of data collection (paper versus palmtop computer). Few differences were detected between the surveys done on palmtops as compared to paper ones, but urban and rural differentials in contraceptive use were less pronounced in the rapid surveys than in the earlier, national survey. This suggests that caution should be exercised in interpreting the disaggregated data in these rapid surveys. In-depth interviews revealed two features of the rapid surveys that were especially popular: the palmtops for their speed of data entry, and the short questionnaire for its "low impact" on a respondent's time. The common belief that computers would disturb respondents was not found to be the case. Even with no computer experience, the interviewers rapidly mastered the new technology.