Evaluating Meta-Learners to Analyze Treatment Heterogeneity in Survival Data: Application to Electronic Health Records of Pediatric Asthma Care in COVID-19 Pandemic

Abstract

An important aspect of precision medicine focuses on characterizing diverse responses to treatment due to unique patient characteristics, also known as heterogeneous treatment effects (HTE) or individualized treatment effects (ITE), and identifying beneficial subgroups with enhanced treatment effects. Estimating HTE with right-censored data in observational studies remains challenging. In this paper, we propose a pseudo-ITE-based framework for analyzing HTE in survival data, which includes a group of meta-learners for estimating HTE, a variable importance metric for identifying predictive variables to HTE, and a data-adaptive procedure to select subgroups with enhanced treatment effects. We evaluate the finite sample performance of the framework under various observational study settings. Furthermore, we applied the proposed methods to analyze the treatment heterogeneity of a written asthma action plan (WAAP) on time-to-ED (Emergency Department) return due to asthma exacerbation using a large asthma electronic health records dataset with visit records expanded from pre- to post-COVID-19 pandemic. We identified vulnerable subgroups of patients with poorer asthma outcomes but enhanced benefits from WAAP and characterized patient profiles. Our research provides valuable insights for healthcare providers on the strategic distribution of WAAP, particularly during disruptive public health crises, ultimately improving the management and control of pediatric asthma.

Keywords: COVID‐19 pandemic; EHR data; heterogeneous treatment effects; meta‐learner; precision asthma care; subgroup analysis.

FIGURE 1

(A) and (B) present Kaplan Meier (KM) curves by WAAP intervention groups and by pandemic periods. (C) and (D) present KM curves of WAAP interaction with the influenza vaccine and age. Log‐rank tests were conducted to compare the survival curves across different categories.

FIGURE 2

The top two panels are the prediction performance under the balanced design of observational studies at $t^{\ast }=$ median survival time. The bottom panel is the attribution score of variables directly contributing to CATE. X, M, DR, R, D, and DEA are six meta‐learners; CSF: causal survival forests; Weib: Weibull regression models. To save computational load, we calculated the attribution score of each method on 100 test samples.

FIGURE 3

Performance of CATE estimation under balanced design of observational studies at the median survival time under covariate‐dependent censoring with censoring rate as 30%. X, M, DR, R, D, and DEA are six meta‐learners; CSF: causal survival forests; Weib: Weibull regression models. Censoring probability was modeled by RSF; propensity score was modeled by RF; conditional survival probability under each treatment group is modeled by RSF. Pseudo‐ITE regression is modeled by RF.

FIGURE 4

Mean treatment difference (MTD) over top $1-q$ subgroup of patients for $q=90\%,80\%,\dots ,10\%$, where $q$'s are the percentiles of predicted CATEs using R‐learner. The horizontal dashed line is the overall MTD in all patients.

FIGURE 5

Mean of the predicted CATE values in each top $(1-q)$ subgroup by top predictors. The color represents the averaged CATE under each top $(1-q)$ subgroup.