A critical analysis of the combined usage of protein localization prediction methods: Increasing the number of independent data sets can reduce the accuracy of predicted mitochondrial localization

Abstract

In the absence of a comprehensive experimentally derived mitochondrial proteome, several bioinformatic approaches have been developed to aid the identification of novel mitochondrial disease genes within mapped nuclear genetic loci. Often, many classifiers are combined to increase the sensitivity and specificity of the predictions. Here we show that the greatest sensitivity and specificity are obtained by using a combination of seven carefully selected classifiers. We also show that increasing the number of independent prediction methods can paradoxically decrease the accuracy of predicting mitochondrial localization. This approach will help to accelerate the identification of new mitochondrial disease genes by providing a principled way for the selection for combination of appropriate prediction methods of mitochondrial localization of proteins.

Fig. 1

Sensitivity of different combinations of prediction tools in rank order. Left hand = detail of the top 100 combinations.

Fig. 2

Range of mean (a) specificity and (b) sensitivity values for different combinations (n = 2047) of the eleven different prediction tools. Horizontal bar = mean, box = standard deviation, whiskers = range.

Fig. 3

The contribution of each prediction tool to mean specificity (a and b); and mean sensitivity (c and d) for 100 training and test runs of the SVM. In this figure, “probability” refers to the proportion of combinations where a specific prediction tool contributed to a result with a given specificity and sensitivity. (a) Heat contour plot showing the probability of a specific tool or data set in contributing to a given level of specificity. Red = high probability, yellow = low probability. (b) Probability of each prediction tool contributing to a given level of specificity for the top 100 combinations. (c) Heat contour plot showing the probability of a specific tool in contributing to a given level of sensitivity. Red = high probability, yellow = low probability. (d) Probability of each prediction tool contributing to a given level of sensitivity for the prediction tools generating the top 100 mean sensitivity values.

Fig. 4

Relationship between sensitivity and specificity for the 2047 different combinations of the prediction tools. Color code = the number of prediction tools incorporated in the SVM prediction. Note the disproportionate axes.