Accuracy of random-forest-based imputation of missing data in the presence of non-normality, non-linearity, and interaction

Abstract

Background: Missing data are common in statistical analyses, and imputation methods based on random forests (RF) are becoming popular for handling missing data especially in biomedical research. Unlike standard imputation approaches, RF-based imputation methods do not assume normality or require specification of parametric models. However, it is still inconclusive how they perform for non-normally distributed data or when there are non-linear relationships or interactions.

Methods: To examine the effects of these three factors, a variety of datasets were simulated with outcome-dependent missing at random (MAR) covariates, and the performances of the RF-based imputation methods missForest and CALIBERrfimpute were evaluated in comparison with predictive mean matching (PMM).

Results: Both missForest and CALIBERrfimpute have high predictive accuracy but missForest can produce severely biased regression coefficient estimates and downward biased confidence interval coverages, especially for highly skewed variables in nonlinear models. CALIBERrfimpute typically outperforms missForest when estimating regression coefficients, although its biases are still substantial and can be worse than PMM for logistic regression relationships with interaction.

Conclusions: RF-based imputation, in particular missForest, should not be indiscriminately recommended as a panacea for imputing missing data, especially when data are highly skewed and/or outcome-dependent MAR. A correct analysis requires a careful critique of the missing data mechanism and the inter-relationships between the variables in the data.

Keywords: Imputation accuracy; Missing data imputation; Random forest.

Fig. 1

Distributions used for covariate X. (a) symmetric distributions (normal and uniform), (b) lognormal distributions, (c) gamma distributions, (d) bimodal distributions (mixture of two normal distributions). The panels display the kernel densities based on 1 million observations randomly sampled from each distribution. $\left[\mathrm{N}\left({\mu }_1,{\sigma }_1^2\right),\mathrm{N}\left({\mu }_1,{\sigma }_1^2\right)\right]$ represents a homogeneous mixture of 50% $\text{Normal}\left({\mu }_1,{\sigma }_1^2\right)$ and 50% $\text{Normal}\left({\mu }_1,{\sigma }_1^2\right)$. For figures with boxplots, the top and bottom 0.025 percentiles were truncated to avoid extreme values in order to facilitate the visual comparison of the boxplots

Fig. 2

Relative bias of the estimated mean of imputed variables for MAR data

Fig. 3

Relative bias of the estimated regression coefficient of imputed variables for MAR data

Fig. 4

Coverage of 95% confidence intervals (with binomial proportion confidence intervals) of the estimated regression coefficients of imputed variables for MAR data

Fig. 5

Lin’s concordance correlation coefficient (CCC) from predictions using models estimated from imputed MAR data

Fig. 6

Scatter plot of Y versus X from imputation results of a randomly selected dataset when X~Gamma(1, 1) in scenario 1 for MAR data. Dashed line is for the true model, solid line is for the estimated model using imputed data. Only imputed observations were shown for direct comparison between imputation methods