The impact of misclassifications and outliers on imputation methods

Abstract

Many imputation methods have been developed over the years and tested mostly under ideal settings. Surprisingly, there is no detailed research on how imputation methods perform when the idealized assumptions about the distribution of data and/or model assumptions are partly not fulfilled. This research looks into the susceptibility of imputation techniques, particularly in relation to outliers, misclassifications, and incorrect model specifications. This is crucial knowledge about how well the methods convince in everyday life because, in reality, conditions are usually not ideal, and model assumptions may not hold. The data may not fit the defined models well. Outliers distort the estimates, and misclassifications reduce the quality of most imputation methods. Several different evaluation measures are discussed, from comparing imputed values with true values or comparing certain statistics, from the performance of classifiers to the variance of estimated parameters. Some well-known imputation methods are compared based on real data and simulations. It turns out that robust conditional imputation methods outperform other methods for real data and simulation settings.

Keywords: 62-08; Missing values; imputation; misclassifications; outliers; robust methods; simulation.

Figure 1.

Results for the Animals data set. True values set to missing (grey triangles), imputed values (red points) and their connecting lines to the true values (dashed gray lines), robust tolerance ellipses of the complete data (ellipses in solid black lines) and after imputation (ellipses in black dashed lines) and the subset of values to be imputed (ellipses in solid red for true values and in dashed red lines for imputed values).

Figure 2.

The biplot for the complete data set (Prestige) is shown in the upper left. The gray dots represent values that are set artificially to missing with MAR. The imputed values from different methods are also shown in gray, while the fully observed observations are again in black.

Figure 3.

NRMSE and MSECOR from the imputation of the Prestige data.

Figure 4.

NRMSE and MSECOR from the imputation of the Bushfire data (in log-scale).

Figure 5.

NRMSE and MSECOR from imputation of the Freedman data.

Figure 6.

NRMSE, MSECOR (both in log10-scale) and false classifications of imputed values from the imputation of the Iris data.

Figure 7.

Results on the root mean squared error of the arithmetic mean estimates based on the simulation setting 1. The thick green line represents the actual method, while the thin light gray lines are used for comparison with other methods.

Figure 8.

Results on the coverage rate of the confidence interval of the arithmetic mean based on the simulation setting 1. The thick green line represents the actual method, while the thin light gray lines are put for comparison with other methods.

Figure 9.

Results on the root mean squared error of the arithmetic mean estimates based on the simulation setting 2. The thick green line represents the actual method, while the thin light gray lines are used for comparison with other methods.

Figure 10.

Results on the coverage rate of the confidence interval of the arithmetic mean based on the simulation setting 1. The thick green line represents the actual method, while the thin light gray lines are used for comparison with other methods.

Figure 11.

F1 score of a random forest classifier for the credit card fraud dataset after including and imputing missing with different imputation methods.

Figure 12.

F1 score of a random forest classifier for the sonar dataset after including and imputing missing with different imputation methods.