Veridical Causal Inference using Propensity Score Methods for Comparative Effectiveness Research with Medical Claims

Abstract

Medical insurance claims are becoming increasingly common data sources to answer a variety of questions in biomedical research. Although comprehensive in terms of longitudinal characterization of disease development and progression for a potentially large number of patients, population-based inference using these datasets require thoughtful modifications to sample selection and analytic strategies relative to other types of studies. Along with complex selection bias and missing data issues, claims-based studies are purely observational, which limits effective understanding and characterization of the treatment differences between groups being compared. All these issues contribute to a crisis in reproducibility and replication of comparative findings using medical claims. This paper offers practical guidance to the analytical process, demonstrates methods for estimating causal treatment effects with propensity score methods for several types of outcomes common to such studies, such as binary, count, time to event and longitudinally-varying measures, and also aims to increase transparency and reproducibility of reporting of results from these investigations. We provide an online version of the paper with readily implementable code for the entire analysis pipeline to serve as a guided tutorial for practitioners. The online version can be accessed at https://rydaro.github.io/. The analytic pipeline is illustrated using a sub-cohort of patients with advanced prostate cancer from the large Clinformatics TM Data Mart Database (OptumInsight, Eden Prairie, Minnesota), consisting of 73 million distinct private payer insurees from 2001-2016.

Keywords: average treatment effect; covariate adjustment; hormone therapy; insurance claims; matching; prostate cancer; reproducibility; sensitivity analysis; veridical data science.

Figure 1:

Comparative Effectiveness Data Analysis Pipeline Flow Diagram. Gold pathway indicates steps done in iteration until acceptable balance is achieved.

Figure 2:

Balance Diagnostics Plot Standardized differences shown for each confounder variable. Vertical dotted lines indicate the desired balance level. Differences shown for the observed data, after matching, and weighting with both calculated propensity scores (logistic regression and CBPS)

Figure 3:

Visualized Sensitivity Analysis Four sensitivity analyses for four ATE estimates are shown. Contours over scatterplot show the entire distribution of ATE and associated p-values for the set of plausible propensity score models. Dashed lines show denoted percentiles cutoffs for this distribution. K denotes number of covariates in shown model, and the dotted line plot shows median ATE and p-value for each set of K covariates from K=1 to K=12. Thick solid line indicates significance threshold of alpha=0.05. E-values for the full model (K=12) are listed in caption.