A comparison of performance of plant miRNA target prediction tools and the characterization of features for genome-wide target prediction

Abstract

Background: Deep-sequencing has enabled the identification of large numbers of miRNAs and siRNAs, making the high-throughput target identification a main limiting factor in defining their function. In plants, several tools have been developed to predict targets, majority of them being trained on Arabidopsis datasets. An extensive and systematic evaluation has not been made for their suitability for predicting targets in species other than Arabidopsis. Nor, these have not been evaluated for their suitability for high-throughput target prediction at genome level.

Results: We evaluated the performance of 11 computational tools in identifying genome-wide targets in Arabidopsis and other plants with procedures that optimized score-cutoffs for estimating targets. Targetfinder was most efficient [89% 'precision' (accuracy of prediction), 97% 'recall' (sensitivity)] in predicting 'true-positive' targets in Arabidopsis miRNA-mRNA interactions. In contrast, only 46% of true positive interactions from non-Arabidopsis species were detected, indicating low 'recall' values. Score optimizations increased the 'recall' to only 70% (corresponding 'precision': 65%) for datasets of true miRNA-mRNA interactions in species other than Arabidopsis. Combining the results of Targetfinder and psRNATarget delivers high true positive coverage, whereas the intersection of psRNATarget and Tapirhybrid outputs deliver highly 'precise' predictions. The large number of 'false negative' predictions delivered from non-Arabidopsis datasets by all the available tools indicate the diversity in miRNAs-mRNA interaction features between Arabidopsis and other species. A subset of miRNA-mRNA interactions differed significantly for features in seed regions as well as the total number of matches/mismatches.

Conclusion: Although, many plant miRNA target prediction tools may be optimized to predict targets with high specificity in Arabidopsis, such optimized thresholds may not be suitable for many targets in non-Arabidopsis species. More importantly, non-conventional features of miRNA-mRNA interaction may exist in plants indicating alternate mode of miRNA target recognition. Incorporation of these divergent features would enable next-generation of algorithms to better identify target interactions.

Figure 1

Schematic representation of the strategy adapted to evaluate smRNA target prediction tool in plants.

Figure 2

Computational time required for each of the tools to predict targets in Arabidopsis transcriptome at their default settings.

Figure 3

Genome-wide evaluation of tools for target-prediction in A. thaliana . (A) Average number of targets predicted by the different tools from the Arabidopsis transcriptome for a miRNA. (B) Number of predictions required by different tools for attaining a true positive rate of 1. Error bars represent standard deviations.

Figure 4

Comparison of ‘precision’ and ‘recall’ rates for prediction by various tools to determine optimal scores for predictions of targets in Arabidopsis dataset. The intersection of ‘precision’ and ‘recall’ designates the optimal score for an algorithm.

Figure 5

Combining outputs of individual tools do not affect the performance of predictions in Arabidopsis. (A) Comparison of true positive and false positive predictions by the top 6 tools for the Arabidospsis dataset. The arrow reveals Targetfinder as returning the maximum number of true positives. Union (B) of results (from Targetfinder and Tapirfasta; Tapirfasta results form a subset of results of Targetfinder), or intersection (C) of results (from Tapirfasta, Tapirhybrid and Target_Prediction) do not improve prediction rates as compared to those returned by Targetfinder alone. Bold and regular numbers represent false positives and true positives respectively.

Figure 6

Evaluation of plant miRNA target prediction tools for identifying true miRNA-mRNA interaction in non-Arabidopsis species. (A) Comparison of ‘precision’ and ‘recall’ of the six plant specific tools to optimize scores for predicting targets in non-Arabidopsis dataset. (B) Comparison of true positive and false positive predictions by 6 tools independently and in-combination. The intersection of psRNATarget and Tapirhybrid (PH, marked with an arrow) delivers the best trade-off between true and false positive rates. Overlap of TP and FN is represented in (C) when the union of Targetfinder and psRNATarget (upper Venn diagram) or the intersection of psRNATarget, Tapirhybrid, and Targetfinder (PHI; lower Venn diagram) of predictions are made. Again, bold and regular numbers represent false positives and true positives respectively.

Figure 7

Relationship between the free energy and the transcript length. Density plots show how the free energy changes with the increase in length of the mRNAs that were used for the prediction of miRNA-targets. The red line represents the loess based curve fitted data points.

Figure 8

Characterization of features of true positive (TP) and false negative (FN) predictions in Arabidopsis and non-Arabidopsis datasets. (A) Distribution of first stretch of miRNA-mRNA targets in TPs and FNs for Arabidopsis and non-Arabidopsis datasets: i) and ii) show the length distribution for the TP and FN miRNAs in Arabidopsis dataset, respectively, while iii) and iv) show the length distributions for TP and FN datasets in non-Arabidopsis dataset, respectively. (B) Comparison of the ‘seed region’ (miRNA region with maximum continuous matches with its targets) in TPs and FNs for Arabidopsis and non-Arabidopsis dataset: i) and ii) are the lengths of seed region distributions for the TP and FN miRNAs in Arabidopsis species, respectively, while iii) and iv) represent the lengths of seed region distributions for TP and FN datasets in non-Arabidopsis datasets respectively. (C) Distribution of the match-mismatch ratio (number of matches divided by the number of mismatches for miRNA with its target sequences) in TPs and FNs for Arabidopsis and non-Arabidopsis datasets. i) and ii) show the match-mismatch ratio for the TP and FN miRNAs in Arabidopsis dataset respectively, while iii) and iv) are match-mismatch ratios of seed region distributions for TP and FN datasets in non-Arabidopsis dataset respectively. (D) Change in entropy with respect to the position of the miRNA:mRNA interaction. Entropy value for each miRNA-mRNA positions for TP predictions in Arabidopsis and non-Arabidopsis datasets and FN predictions for non-Arabidopsis datasets are plotted in black, red and blue respectively.