Implementation and applications of bootstrap methods for the National Immunization Survey

Abstract

In complex probability sample surveys, numerous adjustments are customarily made to the survey weights to reduce potential bias in survey estimates. These adjustments include sampling design (SD) weight adjustments, which account for features of the sampling plan, and non-sampling design (NSD) weight adjustments, which account for non-sampling errors and other effects. Variance estimates prepared from complex survey data customarily account for SD weight adjustments, but rarely account for all NSD weight adjustments. As a result, variance estimates may be biased and standard confidence intervals may not achieve their nominal coverage levels. We describe the implementation of the bootstrap method to account for the SD and NSD weight adjustments for complex survey data. Using data from the National Immunization Survey (NIS), we illustrate the use of the bootstrap (i). for evaluating the use of standard confidence intervals that use Taylor series approximations to variance estimators that do not account for NSD weight adjustments, (ii). for obtaining confidence intervals for ranks estimated from weighted survey data, and (iii). for evaluating the predictive power of logistic regressions using receiver operating characteristic curve analyses that account for the SD and NSD adjustments made to the survey weights.