A systematic approach towards missing lab data in electronic health records: A case study in non-small cell lung cancer and multiple myeloma

Abstract

Real-world data derived from electronic health records often exhibit high levels of missingness in variables, such as laboratory results, presenting a challenge for statistical analyses. We developed a systematic workflow for gathering evidence of different missingness mechanisms and performing subsequent statistical analyses. We quantify evidence for missing completely at random (MCAR) or missing at random (MAR), mechanisms using Hotelling's multivariate t-test, and random forest classifiers, respectively. We further illustrate how to apply sensitivity analyses using the not at random fully conditional specification procedure to examine changes in parameter estimates under missing not at random (MNAR) mechanisms. In simulation studies, we validated these diagnostics and compared analytic bias under different mechanisms. To demonstrate the application of this workflow, we applied it to two exemplary case studies with an advanced non-small cell lung cancer and a multiple myeloma cohort derived from a real-world oncology database. Here, we found strong evidence against MCAR, and some evidence of MAR, implying that imputation approaches that attempt to predict missing values by fitting a model to observed data may be suitable for use. Sensitivity analyses did not suggest meaningful departures of our analytic results under potential MNAR mechanisms; these results were also in line with results reported in clinical trials.

FIGURE 1

M‐graph representations of the four missingness mechanisms considered, with $\mathrm{la}{\mathrm{b}}_{\mathrm{obs}}$ representing observed laboratory values subject to missingness. (a) MCAR: Missingness ($M_{\mathrm{lab}}$) is independent of the true laboratory values ($\mathrm{lab}$) and any other variables $X$. (b) MAR: Missingness ($M_{\mathrm{lab}}$) depends on observed variables $\mathsf{X}$ but is independent of true laboratory values ($\mathrm{lab}$). (c) MNAR‐unmeasured: Missingness ($M_{\mathrm{lab}}$) depends on unobserved variables $U$ but is independent of true laboratory values ($\mathrm{lab}$). (d) MNAR‐value: Missingness ($M_{\mathrm{lab}}$) depends on true laboratory values ($\mathrm{lab}$). A directed edge represents a causal effect of a variable on another. MAR, missing at random; MCAR, missing completely at random; MNAR, missing not at random.

FIGURE 2

Illustration of systematic workflow to diagnose potential missingness mechanisms. AUC, area under the receiver operating characteristic curve; H0, null hypothesis; MAR, missing at random; MCAR, missing completely at random; MNAR, missing not at random; NARFCS, not at random fully conditional specification; SMD, standardized mean difference.

FIGURE 3

Top: Distributions of L1 bias (sum of absolute biases across all hazard ratios estimated) in simulation studies, by missingness mechanism. Middle: Distributions of absolute bias for each hazard ratio estimated (laboratory, treatment, and covariate) in simulation studies, by missingness mechanism. Bottom: Coverage probabilities of 95% confidence intervals for each hazard ratio estimated (laboratory, treatment, and covariate) in simulation studies, by missingness mechanism. MAR, missing at random; MCAR, missing completely at random; MNAR, missing not at random.

FIGURE 4

AUC diagnostic results on real‐world cohorts. Covariates used in aNSCLC cohort: Age at index date, gender, index year, histology, group stage, smoking status, birth year, race/ethnicity, region, ECOG, age at diagnosis, age at advanced diagnosis, time from initial diagnosis to index, time from advanced diagnosis to index date, time from index date to end of follow‐up, censoring indicator, PD‐L1 status. Covariates used in MM cohort: Age at index date (1 L treatment start), ECOG at index date, gender, ISS stage, index year (calendar year of 1 L treatment start), practice type, race/ethnicity, region, time from diagnosis to index date, time from index date to end of follow‐up, censoring indicator, line of therapy. aNSCLC, advanced non‐small cell lung cancer; AUC, area under the receiver operating characteristic curve; ECOG, Eastern Cooperative Oncology Group; ISS, International Staging System; MM, multiple myeloma.

FIGURE 5

NARFCS sensitivity analysis results for real world analyses in NSCLC and MM. $\delta =0$ indicates HR estimated under multiple imputation without any sensitivity adjustment. $\delta >0$ indicates a shift in the imputation model where missing laboratory values are more likely to be normal than those observed; $\delta <0$ indicates a shift in the imputation model where missing laboratory values are more likely to be abnormal than those observed. Y‐axis displays estimated HR with 95% confidence interval. HR, hazard ratio; MM, multiple myeloma; NARFCS, not at random fully conditional specification; NSCLC, non‐small cell lung cancer.