Bayesian penalized spline model-based inference for finite population proportion in unequal probability sampling

Abstract

We propose a Bayesian Penalized Spline Predictive (BPSP) estimator for a finite population proportion in an unequal probability sampling setting. This new method allows the probabilities of inclusion to be directly incorporated into the estimation of a population proportion, using a probit regression of the binary outcome on the penalized spline of the inclusion probabilities. The posterior predictive distribution of the population proportion is obtained using Gibbs sampling. The advantages of the BPSP estimator over the Hájek (HK), Generalized Regression (GR), and parametric model-based prediction estimators are demonstrated by simulation studies and a real example in tax auditing. Simulation studies show that the BPSP estimator is more efficient, and its 95% credible interval provides better confidence coverage with shorter average width than the HK and GR estimators, especially when the population proportion is close to zero or one or when the sample is small. Compared to linear model-based predictive estimators, the BPSP estimators are robust to model misspecification and influential observations in the sample.

Keywords: Bayesian analysis; Binary data; Penalized spline regression; Probability proportional to size; Survey samples.

Figure 1

Two simulated artificial populations (N = 2,000)

Figure 2

A random pps sample from the EXP case (n = 200, N = 2,000): (a) scatter plot of Z; the three grey lines are the superpopulation 10^th, 50^th, and 90^th percentiles, respectively. (b) black circles are observed units of binary survey variable Y in the sample, defined as Y = I (Z ≤ 10^th percentile); the grey solid and dashed curves are posterior means of Pr (Y_i = 1|π_i) and 95% credible intervals, respectively, simulated based on a probit p-spline model on π; and the black curve is the superpopulation Pr(Y_i = 1|π_i). (c) similar to (b), but with Y = I (Z ≤ 50^th percentile). (d) similar to (b), but with Y = I (Z ≤ 90^th percentile)

Figure 3

Box plots of the probabilities of inclusion for two sample sizes in the tax auditing example

Figure 4

Predictions based on pps samples only in the tax auditing example, X-axis: inclusion probabilities π, Y-axis: P(Y = 1|π); black dots are the true P(Y = 1|π) within each percentile of π; grey curves are ten realizations of the posterior means of P(Y = 1|π). The prediction models are (a) probit linear p-spline regression, (b) linear probit regression, (c) quadratic probit regression

Figure 5

Predictions based on the combined data of pps samples and the observations sampled with certainty in the tax auditing example, X-axis: inclusion probabilities π, Y-axis: P(Y = 1|π); black dots are the true P(Y = 1|π) within each percentile of π; grey curves are ten realizations of the posterior mean of P(Y = 1|π). The prediction models are (a) probit linear p-spline regression, (b) linear probit regression, (c) quadratic probit regression