Generalizing trial evidence to target populations in non-nested designs: Applications to AIDS clinical trials

Abstract

Comparative effectiveness evidence from randomized trials may not be directly generalizable to a target population of substantive interest when, as in most cases, trial participants are not randomly sampled from the target population. Motivated by the need to generalize evidence from two trials conducted in the AIDS Clinical Trials Group (ACTG), we consider weighting, regression and doubly robust estimators to estimate the causal effects of HIV interventions in a specified population of people living with HIV in the USA. We focus on a non-nested trial design and discuss strategies for both point and variance estimation of the target population average treatment effect. Specifically in the generalizability context, we demonstrate both analytically and empirically that estimating the known propensity score in trials does not increase the variance for each of the weighting, regression and doubly robust estimators. We apply these methods to generalize the average treatment effects from two ACTG trials to specified target populations and operationalize key practical considerations. Finally, we report on a simulation study that investigates the finite-sample operating characteristics of the generalizability estimators and their sandwich variance estimators.

Keywords: causal inference; double robustness; generalizability; internal validity; inverse probability weighting; propensity score; sampling score.

FIGURE 1

Histograms of estimated sampling scores for each of the four generalizability analyses

FIGURE 2

Boxplots of the standard mean differences (SMDs) of all covariates (and interaction terms) for the unweighted trial sample and inverse probability of participation weighted trial sample for each of the four analyses

FIGURE 3

PATE estimates and 95% confidence intervals. The three sets of results correspond to generalizability estimators that use (i) the true propensity score (PS method = True); (ii) the estimated propensity score using a main-effects logistic model (PS method = Main); and (iii) the estimated propensity score using a full logistic model (PS method=Full). The dashed lines indicate results for the within-trial SATE