Random Forest Missing Data Algorithms

Abstract

Random forest (RF) missing data algorithms are an attractive approach for imputing missing data. They have the desirable properties of being able to handle mixed types of missing data, they are adaptive to interactions and nonlinearity, and they have the potential to scale to big data settings. Currently there are many different RF imputation algorithms, but relatively little guidance about their efficacy. Using a large, diverse collection of data sets, imputation performance of various RF algorithms was assessed under different missing data mechanisms. Algorithms included proximity imputation, on the fly imputation, and imputation utilizing multivariate unsupervised and supervised splitting-the latter class representing a generalization of a new promising imputation algorithm called missForest. Our findings reveal RF imputation to be generally robust with performance improving with increasing correlation. Performance was good under moderate to high missingness, and even (in certain cases) when data was missing not at random.

Keywords: Correlation; Imputation; Machine Learning; Missingness; Splitting (random; multivariate; univariate; unsupervised).

Figure 1

Summary values for the 60 data sets used in the large scale RF missing data experiment. The last panel displays the log-information, I = log₁₀(n/p), for each data set.

Figure 2

ANOVA effect size for the log-information, I = log₁₀(n/p), and correlation, ρ (defined as in (3)), from a linear regression using log relative imputation error, log₁₀(ℰ_R(ℐ)), as the response. In addition to I and ρ, dependent variables in the regression included type of RF procedure used. ANOVA effect sizes are the estimated coefficients of the standardized variable (standardized to have mean zero and variance 1). This demonstrates the importance of correlation in assessing imputation performance.

Figure 3

Relative imputation error, ℰ_R(ℐ), stratified and averaged by level of correlation of a data set. Procedures are: RF_otf, ${RF}_{\mathit{otf}}^{(5)}$ (on the fly imputation with 1 and 5 iterations); RF_otfR, ${RF}_{\mathit{otfR}}^{(5)}$ (similar to RF_otf and ${RF}_{\mathit{otf}}^{(5)}$ but using pure random splitting); RF_unsv, ${RF}_{\mathit{unsv}}^{(5)}$ (multivariate unsupervised splitting with 1 and 5 iterations); RF_prx, ${RF}_{\mathit{prx}}^{(5)}$ (proximity imputation with 1 and 5 iterations); RF_prxR, ${RF}_{\mathit{prxR}}^{(5)}$ (same as RF_prx and ${RF}_{\mathit{prx}}^{(5)}$ but using pure random splitting); mRF_0.25, mRF_0.05, mRF (mForest imputation, with 25%, 5% and 1 variable(s) used as the response); KNN (k-nearest neighbor imputation).

Figure 4

Mean relative imputation error ± standard deviation from simulations under different sample size values n = 100, 200, 500, 1000, 2000.

Figure 5

Log of computing time for a procedure versus log-computational complexity of a data set, c = log₁₀(np).

Figure 6

Relative log-computing time (relative to KNN) versus log-computational complexity of a data set, c = log₁₀(np).