Performance of Cross-Validated Targeted Maximum Likelihood Estimation

Abstract

Background: Advanced methods for causal inference, such as targeted maximum likelihood estimation (TMLE), require specific convergence rates and the Donsker class condition for valid statistical estimation and inference. In situations where there is no differentiability due to data sparsity or near-positivity violations, the Donsker class condition is violated. In such instances, the bias of the targeted estimand is inflated, and its variance is anti-conservative, leading to poor coverage. Cross-validation of the TMLE algorithm (CVTMLE) is a straightforward, yet effective way to ensure efficiency, especially in settings where the Donsker class condition is violated, such as random or near-positivity violations. We aim to investigate the performance of CVTMLE compared to TMLE in various settings.

Methods: We utilized the data-generating mechanism described in Leger et al. (2022) to run a Monte Carlo experiment under different Donsker class violations. Then, we evaluated the respective statistical performances of TMLE and CVTMLE with different super learner libraries, with and without regression tree methods.

Results: We found that CVTMLE vastly improves confidence interval coverage without adversely affecting bias, particularly in settings with small sample sizes and near-positivity violations. Furthermore, incorporating regression trees using standard TMLE with ensemble super learner-based initial estimates increases bias and reduces variance, leading to invalid statistical inference.

Conclusions: We show through simulations that CVTMLE is much less sensitive to the choice of the super learner library and thereby provides better estimation and inference in cases where the super learner library uses more flexible candidates and is prone to overfitting.

Keywords: Donsker class condition; causal inference; data sparsity; epidemiology; near‐positivity violation; observational studies; targeted maximum likelihood estimation.

FIGURE 1

Process map of cross‐validated targeted maximum likelihood estimation.

FIGURE 2

Relative bias and coverage of all TMLE and CVTMLE approaches under data‐generating mechanisms 1–4.

FIGURE 3

Relative bias and coverage of all TMLE and CVTMLE approaches under data‐generating mechanisms 5–8.

FIGURE 4

Decision tree for the appropriate choice of method given the scenarios (i.e., near‐positivity violation, sample size) that can cause the lack of differentiability of the influence curve and potentially violate the Donsker class condition.

FIGURE A1

Probability of the outcome given the exposure and Z1 (variable creating near‐positivity violations), stratified by prevalence of the exposure (i.e., 50% or 80%) and presence of extrapolation issue (i.e., none or high). (A) is 50% prevalence of the exposure with no extrapolation issue. (B) is 80% prevalence of the exposure with no extrapolation issue. (C) is a 50% prevalence of the exposure with an extrapolation issue. (D) is an 80% prevalence of the exposure with an extrapolation issue.