On the Use of Auxiliary Variables in Multilevel Regression and Poststratification

Abstract

Multilevel regression and poststratification (MRP) is a popular method for addressing selection bias in subgroup estimation, with broad applications across fields from social sciences to public health. In this paper, we examine the inferential validity of MRP in finite populations, exploring the impact of poststratification and model specification. The success of MRP relies heavily on the availability of auxiliary information that is strongly related to the outcome. To enhance the fitting performance of the outcome model, we recommend modeling the inclusion probabilities conditionally on auxiliary variables and incorporating flexible functions of estimated inclusion probabilities as predictors in the mean structure. We present a statistical data integration framework that offers robust inferences for probability and nonprobability surveys, addressing various challenges in practical applications. Our simulation studies indicate the statistical validity of MRP, which involves a tradeoff between bias and variance, with greater benefits for subgroup estimates with small sample sizes, compared to alternative methods. We have applied our methods to the Adolescent Brain Cognitive Development (ABCD) Study, which collected information on children across 21 geographic locations in the U.S. to provide national representation, but is subject to selection bias as a nonprobability sample. We focus on the cognition measure of diverse groups of children in the ABCD study and show that the use of auxiliary variables affects the findings on cognitive performance.

Keywords: data integration; model-based; nonprobability sample; robust inference; selection/nonresponse bias.

Figure 1:

Outputs of overall and subgroup mean estimates when the outcome models and adjustment factors of all methods are correctly specified. RMSE: root mean squared error, DR: doubly robust estimator, IPW: inverse propensity weighting estimator, GREG: generalized regression estimator, MRP: multilevel regression and poststratification, MRP-INT: an integrated MRP by adding estimated inclusion probabilities as predictors in the outcome models and predicting all population cells with known population cell sizes, MRP-P: MRP that uses all population cells with known population cell sizes, and MRP-R: MRP that uses available cells in the reference probability sample with unknown population cell sizes.

Figure 2:

Outputs of overall and subgroup mean estimates when the outcome models and adjustment factors of all methods are incorrectly specified. RMSE: root mean squared error, DR: doubly robust estimator, IPW: inverse propensity weighting estimator, GREG: generalized regression estimator, MRP: multilevel regression and poststratification, MRP-INT: an integrated MRP by adding estimated inclusion probabilities as predictors in the outcome models and predicting all population cells with known population cell sizes, MRP-P: MRP that uses all population cells with known population cell sizes, and MRP-R: MRP that uses available cells in the reference probability sample with unknown population cell sizes.

Figure 3:

Finite population inferences of average cognition test scores by groups, with seven auxiliary variables. The error bars are 95% confidence intervals. HH: household, IPW: inverse propensity weighting estimator, MRP: multilevel regression and poststratification, MRP-INT: an integrated MRP by adding estimated inclusion probabilities as predictors in the outcome models and predicting all population cells with known population cell sizes, MRP-P: MRP that uses all population cells with known population cell sizes, and MRP-R: MRP that uses available cells in the reference probability sample with unknown population cell sizes.

Figure 4:

Finite population inferences of average cognition test scores by groups. The outcome model has five auxiliary variables. The error bars are 95% confidence intervals. HH: household, IPW: inverse propensity weighting estimator, MRP: multilevel regression and poststratification, MRP-INT: an integrated MRP by adding estimated inclusion probabilities as predictors in the outcome models and predicting all population cells with known population cell sizes, MRP-P: MRP that uses all population cells with known population cell sizes, and MRP-R: MRP that uses available cells in the reference probability sample with unknown population cell sizes.