On the use of propensity scores in case of rare exposure

Abstract

Background: Observational post-marketing assessment studies often involve evaluating the effect of a rare treatment on a time-to-event outcome, through the estimation of a marginal hazard ratio. Propensity score (PS) methods are the most used methods to estimate marginal effect of an exposure in observational studies. However there is paucity of data concerning their performance in a context of low prevalence of exposure.

Methods: We conducted an extensive series of Monte Carlo simulations to examine the performance of the two preferred PS methods, known as PS-matching and PS-weighting to estimate marginal hazard ratios, through various scenarios.

Results: We found that both PS-weighting and PS-matching could be biased when estimating the marginal effect of rare exposure. The less biased results were obtained with estimators of average treatment effect in the treated population (ATT), in comparison with estimators of average treatment effect in the overall population (ATE). Among ATT estimators, PS-weighting using ATT weights outperformed PS-matching. These results are illustrated using a real observational study.

Conclusions: When clinical objectives are focused on the treated population, applied researchers are encouraged to estimate ATT with PS-weighting for studying the relative effect of a rare treatment on time-to-event outcomes.

Keywords: Hazard ratio; Monte Carlo simulations; Observational studies; Pharmacoepidemiology; Propensity scores; Rare exposure.

Fig. 1

Directed acyclic graph corresponding to the data-generating algorithm

Fig. 2

Effect of exposure prevalence. Bias of exposure effect, variability ratio, 1 - coverage and RMSE according to prevalence p of exposure and mean sample size, for one continuous and one dichotomous confounder, σ _U,B=σ _U,C=0.3, σ _B,C=0, $exp\left({\delta }_B\right)=exp\left({\delta }_C\right)=1.5$, r _c=50 % and H R=1, with weighting by inverse of PS using ATE and ATT weights and PS-matching

Fig. 3

Effect of theoretical hazard ratio. Bias of exposure effect, variability ratio, 1 - coverage and RMSEw according to theoretical exposure effect (HR) and mean sample size, for one continuous and one dichotomous confounder, σ _U,B=σ _U,C=0.3, σ _B,C=0, $exp\left({\delta }_B\right)=exp\left({\delta }_C\right)=1.5$, r _c=50 % and p=5 %, with weighting by inverse of PS using ATE and ATT weights and PS-matching

Fig. 4

Effect of strength of confounding. Bias of exposure effect, variability ratio, 1 - coverage and RMSE according to strength of confounding and mean sample size, for one continuous and one dichotomous confounder, σ _B,C=0, H R=1, r _c=50 % and p=5 %, with weighting by inverse of PS using ATE and ATT weights and PS-matching

Fig. 5

Effect of the number of confounders. Bias of exposure effect, variability ratio, 1 - coverage and RMSE according to number of confounders (2 or 4 confounders) and mean sample size, for σ _U,B=σ _U,C=0.3, σ _B,C=0, $exp\left({\delta }_B\right)=exp\left({\delta }_C\right)=1.5$, H R=1, r _c=50 % and p=5 %, with weighting by inverse of PS using ATE and ATT weights and PS-matching

Fig. 6

Effect of censoring rate. Bias of exposure effect, variability ratio, 1 - coverage and RMSE according to censoring rate (r _c) and mean sample size, for one continuous and one dichotomous confounder, σ _U,B=σ _U,C=0.3, σ _B,C=0, $exp\left({\delta }_B\right)=exp\left({\delta }_C\right)=1.5$, H R=1 and p=5 %, with weighting by inverse of PS using ATE and ATT weights and PS-matching

Fig. 7

Effect of correlation between covariates. Bias of exposure effect, variability ratio, 1 - coverage and RMSE according to correlation between covariates B and C (σ _B,C) and mean sample size, for one continuous and one dichotomous confounder, σ _U,B=σ _U,C=0.3, $exp\left({\delta }_B\right)=exp\left({\delta }_C\right)=1.5$, H R=1 and p=5 %, with weighting by inverse of PS using ATE and ATT weights and PS-matching

Fig. 8

Imbalances in the REACH cohort, defined as the standardized means differences of covariate values between the two treatment groups. Solid black line represents an absolute standardized difference of 10 %

Fig. 9

Real observational dataset illustration. Bias of TZD effect estimation in the REACH cohort, using PS-matching and PS-weighting approaches, according to prevalence p and mean sample size