Optimal caliper widths for propensity-score matching when estimating differences in means and differences in proportions in observational studies

Abstract

In a study comparing the effects of two treatments, the propensity score is the probability of assignment to one treatment conditional on a subject's measured baseline covariates. Propensity-score matching is increasingly being used to estimate the effects of exposures using observational data. In the most common implementation of propensity-score matching, pairs of treated and untreated subjects are formed whose propensity scores differ by at most a pre-specified amount (the caliper width). There has been a little research into the optimal caliper width. We conducted an extensive series of Monte Carlo simulations to determine the optimal caliper width for estimating differences in means (for continuous outcomes) and risk differences (for binary outcomes). When estimating differences in means or risk differences, we recommend that researchers match on the logit of the propensity score using calipers of width equal to 0.2 of the standard deviation of the logit of the propensity score. When at least some of the covariates were continuous, then either this value, or one close to it, minimized the mean square error of the resultant estimated treatment effect. It also eliminated at least 98% of the bias in the crude estimator, and it resulted in confidence intervals with approximately the correct coverage rates. Furthermore, the empirical type I error rate was approximately correct. When all of the covariates were binary, then the choice of caliper width had a much smaller impact on the performance of estimation of risk differences and differences in means.

Figure 1

Caliper width and reduction in bias: risk differences.

Figure 2

Caliper width and MSE: risk differences.

Figure 3

Caliper width and coverage of 95% confidence intervals: risk differences.

Figure 4

Caliper width and Type 1 error rates.

Figure 5

Caliper width and reduction in blas: difference in means.

Figure 6

Caliper width and MSE: difference in means.

Figure 7

Caliper width and coverage of 95% confidence intervals: difference in means.

Figure 8

Relationship between caliper width and estimated treatment effect in case study.