Skip to main page content
U.S. flag

An official website of the United States government

Dot gov

The .gov means it’s official.
Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you’re on a federal government site.

Https

The site is secure.
The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

Access keys NCBI Homepage MyNCBI Homepage Main Content Main Navigation
Comparative Study
. 2018:23:192-203.

Data-driven advice for applying machine learning to bioinformatics problems

Randal S Olson  1 , William La Cava, Zairah Mustahsan, Akshay Varik, Jason H Moore
Affiliations
Comparative Study

Data-driven advice for applying machine learning to bioinformatics problems

Randal S Olson et al. Pac Symp Biocomput. 2018.

Abstract

As the bioinformatics field grows, it must keep pace not only with new data but with new algorithms. Here we contribute a thorough analysis of 13 state-of-the-art, commonly used machine learning algorithms on a set of 165 publicly available classification problems in order to provide data-driven algorithm recommendations to current researchers. We present a number of statistical and visual comparisons of algorithm performance and quantify the effect of model selection and algorithm tuning for each algorithm and dataset. The analysis culminates in the recommendation of five algorithms with hyperparameters that maximize classifier performance across the tested problems, as well as general guidelines for applying machine learning to supervised classification problems.

PubMed Disclaimer

Figures

Fig. 1
Fig. 1
Average ranking of the ML algorithms over all datasets. Error bars indicate the 95% confidence interval.
Fig. 2
Fig. 2
Heat map showing the percentage out of 165 datasets a given algorithm outperforms another algorithm in terms of best accuracy on a problem. The algorithms are ordered from top to bottom based on their overall performance on all problems. Two algorithms are considered to have the same performance on a problem if they achieved an accuracy within 1% of each other.
Fig. 3
Fig. 3
Improvement in 10-fold CV accuracy by tuning each ML algorithm’s parameters instead of using the default parameters from scikit-learn.
Fig. 4
Fig. 4
Improvement in 10-fold CV accuracy by model selection and tuning, relative to the average performance on each dataset.
Fig. 5
Fig. 5
Hierarchical clustering of ML algorithms by accuracy rankings across datasets.
See this image and copyright information in PMC

References

    1. Bhaskar H, Hoyle DC, Singh S. Computers in Biology and Medicine. 2006;36:1104. Intelligent Technologies in Medicine and Bioinformatics. - PubMed
    1. McKinney BA, Reif DM, Ritchie MD, Moore JH. Applied Bioinformatics. 2006;5:77. - PMC - PubMed
    1. Liu Y, Gadepalli K, Norouzi M, Dahl GE, Kohlberger T, Boyko A, Venugopalan S, Timofeev A, Nelson PQ, Corrado GS, Hipp JD, Peng L, Stumpe MC. Detecting cancer metastases on gigapixel pathology images. 2017 arXiv e-print https://arxiv.org/abs/1703.02442.
    1. King RD, Feng C, Sutherland A. Applied Artificial Intelligence an International Journal. 1995;9:289.
    1. Tan AC, Gilbert D. An empirical comparison of supervised machine learning techniques in bioinformatics. Proceedings of the First Asia-Pacific Bioinformatics Conference on Bioinformatics 2003. 2003;19

MeSH terms