ESTIMATING TREATMENT EFFECTS ON HEALTHCARE COSTS UNDER EXOGENEITY: IS THERE A 'MAGIC BULLET'?

Abstract

Methods for estimating average treatment effects, under the assumption of no unmeasured confounders, include regression models; propensity score adjustments using stratification, weighting, or matching; and doubly robust estimators (a combination of both). Researchers continue to debate about the best estimator for outcomes such as health care cost data, as they are usually characterized by an asymmetric distribution and heterogeneous treatment effects,. Challenges in finding the right specifications for regression models are well documented in the literature. Propensity score estimators are proposed as alternatives to overcoming these challenges. Using simulations, we find that in moderate size samples (n= 5000), balancing on propensity scores that are estimated from saturated specifications can balance the covariate means across treatment arms but fails to balance higher-order moments and covariances amongst covariates. Therefore, unlike regression model, even if a formal model for outcomes is not required, propensity score estimators can be inefficient at best and biased at worst for health care cost data. Our simulation study, designed to take a 'proof by contradiction' approach, proves that no one estimator can be considered the best under all data generating processes for outcomes such as costs. The inverse-propensity weighted estimator is most likely to be unbiased under alternate data generating processes but is prone to bias under misspecification of the propensity score model and is inefficient compared to an unbiased regression estimator. Our results show that there are no 'magic bullets' when it comes to estimating treatment effects in health care costs. Care should be taken before naively applying any one estimator to estimate average treatment effects in these data. We illustrate the performance of alternative methods in a cost dataset on breast cancer treatment.

Figure 1

Initial Simulation Results (averaged over 1000 replicates): (a) Distribution of propensity scores by treatment groups; (b) Mean X’s by treatment status over estimated propensity score; (c) Std. Deviation of X’s by treatment status over estimated propensity score; (d) Correlation between X’s by treatment status over estimated propensity score.

Figure 2

Probability densities of Y under each data generating process, where Y was scaled so that E(Y) = 1in each case.

Figure 3

(a) Distribution of estimated propensity scores by treatment groups; (b) Imbalances in mean and standard deviation of Charlson Index score (ChSc) and the correlation between CHSc and median Household income between treatment groups at specific values (ranges in the case of correlation) of estimated propensity scores.

Figure 4

Analysis on cost of breast cancer treatments. Profile of residuals over the deciles of linear predictors for log-GLM and EEE methods and the corresponding doubly robust methods log-GLM-DR and EEE-DR.