Matching phenotypes to whole genomes: Lessons learned from four iterations of the personal genome project community challenges

Binghuang Cai¹, Biao Li², Nikki Kiga¹, Janita Thusberg², Timothy Bergquist¹, Yun-Ching Chen³, Noushin Niknafs³, Hannah Carter⁴, Collin Tokheim³, Violeta Beleva-Guthrie³, Christopher Douville³, Rohit Bhattacharya⁵, Hui Ting Grace Yeo³, Jean Fan³, Sohini Sengupta³, Dewey Kim³, Melissa Cline⁶, Tychele Turner⁷, Mark Diekhans⁶, Jan Zaucha^{8

9}, Lipika R Pal¹⁰, Chen Cao^{10

11}, Chen-Hsin Yu^{10

11}, Yizhou Yin^{10

11}, Marco Carraro¹², Manuel Giollo^{12

13}, Carlo Ferrari¹³, Emanuela Leonardi¹⁴, Silvio C E Tosatto^{12

15}, Jason Bobe¹⁶, Madeleine Ball¹⁶, Roger A Hoskins¹⁷, Susanna Repo¹⁸, George Church¹⁶, Steven E Brenner¹⁷, John Moult^{10

19}, Julian Gough⁹, Mario Stanke²⁰, Rachel Karchin^{3

21}, Sean D Mooney¹

Affiliations

¹ Department of Biomedical Informatics & Medical Education, University of Washington School of Medicine, Seattle, Washington.
² The Buck Institute for Research on Aging, Novato, California.
³ Department of Biomedical Engineering and Institute for Computational Medicine, Johns Hopkins University, Baltimore, Maryland.
⁴ Department of Medicine, Division of Medical Genetics, Institute for Genomic Medicine and Moores Cancer Center, University of California San Diego, La Jolla, Califonia.
⁵ Department of Computer Science, Institute for Computational Medicine, Johns Hopkins University, Baltimore, Maryland.
⁶ Department of Biomolecular Engineering, University of California, Santa Cruz, California.
⁷ McKusick-Nathans Institute of Genetic Medicine, Johns Hopkins University, Baltimore, Maryland.
⁸ Department of Computer Science, University of Bristol, Bristol, UK.
⁹ Bristol Centre for Complexity Sciences, University of Bristol, Bristol, UK.
¹⁰ Institute for Bioscience and Biotechnology Research, University of Maryland, Rockville, Maryland.
¹¹ Computational Biology, Bioinformatics and Genomics, Biological Sciences Graduate Program, University of Maryland, College Park, Maryland.
¹² Department of Biomedical Sciences, University of Padova, Padova, Italy.
¹³ Department of Information Engineering, University of Padova, Padova, Italy.
¹⁴ Department of Woman and Child Health, University of Padova, Padova, Italy.
¹⁵ CNR Neuroscience Institute, Padova, Italy.
¹⁶ PersonalGenomes.org, Boston, Massachusetts.
¹⁷ Department of Plant and Microbial Biology, University of California, Berkeley, California.
¹⁸ European Bioinformatics Institute, Hinxton, UK.
¹⁹ Department of Cell Biology and Molecular Genetics, University of Maryland, College Park, Maryland.
²⁰ Institute of Mathematics and Computer Science, University of Greifswald, Greifswald, Germany.
²¹ Department of Oncology, The Johns Hopkins Medical Institutions, Baltimore, Maryland.

PMID: 28544481
PMCID: PMC5645203
DOI: 10.1002/humu.23265

Matching phenotypes to whole genomes: Lessons learned from four iterations of the personal genome project community challenges

Binghuang Cai et al. Hum Mutat. 2017 Sep.

. 2017 Sep;38(9):1266-1276.

doi: 10.1002/humu.23265. Epub 2017 Jun 19.

Authors

Affiliations

¹ Department of Biomedical Informatics & Medical Education, University of Washington School of Medicine, Seattle, Washington.
² The Buck Institute for Research on Aging, Novato, California.
³ Department of Biomedical Engineering and Institute for Computational Medicine, Johns Hopkins University, Baltimore, Maryland.
⁴ Department of Medicine, Division of Medical Genetics, Institute for Genomic Medicine and Moores Cancer Center, University of California San Diego, La Jolla, Califonia.
⁵ Department of Computer Science, Institute for Computational Medicine, Johns Hopkins University, Baltimore, Maryland.
⁶ Department of Biomolecular Engineering, University of California, Santa Cruz, California.
⁷ McKusick-Nathans Institute of Genetic Medicine, Johns Hopkins University, Baltimore, Maryland.
⁸ Department of Computer Science, University of Bristol, Bristol, UK.
⁹ Bristol Centre for Complexity Sciences, University of Bristol, Bristol, UK.
¹⁰ Institute for Bioscience and Biotechnology Research, University of Maryland, Rockville, Maryland.
¹¹ Computational Biology, Bioinformatics and Genomics, Biological Sciences Graduate Program, University of Maryland, College Park, Maryland.
¹² Department of Biomedical Sciences, University of Padova, Padova, Italy.
¹³ Department of Information Engineering, University of Padova, Padova, Italy.
¹⁴ Department of Woman and Child Health, University of Padova, Padova, Italy.
¹⁵ CNR Neuroscience Institute, Padova, Italy.
¹⁶ PersonalGenomes.org, Boston, Massachusetts.
¹⁷ Department of Plant and Microbial Biology, University of California, Berkeley, California.
¹⁸ European Bioinformatics Institute, Hinxton, UK.
¹⁹ Department of Cell Biology and Molecular Genetics, University of Maryland, College Park, Maryland.
²⁰ Institute of Mathematics and Computer Science, University of Greifswald, Greifswald, Germany.
²¹ Department of Oncology, The Johns Hopkins Medical Institutions, Baltimore, Maryland.

PMID: 28544481
PMCID: PMC5645203
DOI: 10.1002/humu.23265

Abstract

The advent of next-generation sequencing has dramatically decreased the cost for whole-genome sequencing and increased the viability for its application in research and clinical care. The Personal Genome Project (PGP) provides unrestricted access to genomes of individuals and their associated phenotypes. This resource enabled the Critical Assessment of Genome Interpretation (CAGI) to create a community challenge to assess the bioinformatics community's ability to predict traits from whole genomes. In the CAGI PGP challenge, researchers were asked to predict whether an individual had a particular trait or profile based on their whole genome. Several approaches were used to assess submissions, including ROC AUC (area under receiver operating characteristic curve), probability rankings, the number of correct predictions, and statistical significance simulations. Overall, we found that prediction of individual traits is difficult, relying on a strong knowledge of trait frequency within the general population, whereas matching genomes to trait profiles relies heavily upon a small number of common traits including ancestry, blood type, and eye color. When a rare genetic disorder is present, profiles can be matched when one or more pathogenic variants are identified. Prediction accuracy has improved substantially over the last 6 years due to improved methodology and a better understanding of features.

Keywords: biomedical informatics; community challenge; critical assessment; genome; genome interpretation; open consent; personal genome project (PGP); phenotype.

PubMed Disclaimer

Figures

**FIGURE 1**
ROC AUC curves of the four submissions of PGP 2011

**FIGURE 2**
PR curves of the four submissions of PGP 2011

**FIGURE 3**
AUC comparison of simulative results and submission for the best submission from Group1 of PGP 2011

**FIGURE 4**
ROC AUC curves of the five submissions of PGP 2015

**FIGURE 5**
PR curves of the five submissions of PGP 2015

**FIGURE 6**
AUC comparison of simulative results and submission for the best submission from Group1 of PGP 2015

See this image and copyright information in PMC

References

1. 1000 Genomes Project Consortium. A map of human genome variation from population-scale sequencing. Nature. 2010;467(7319):1061–1073. - PMC - PubMed
1. Adzhubei I, Jordan DM, Sunyaev SR. Predicting functional effect of human missense mutations using PolyPhen-2. Current Protocols in Human Genetics. 2013;(Unit7.20) Chapter 7. - PMC - PubMed
1. Alexander DH, Novembre J, Lange K. Fast model-based estimation of ancestry in unrelated individuals. Genome Research. 2009;19:9. - PMC - PubMed
1. Anstee DJ. Red cell genotyping and the future of pretransfusion testing. Blood. 2009;114(2):248–256. - PubMed
1. Ball MP, Bobe JR, Chou MF, Clegg T, Estep PW, Lunshof JE, Church GM. Harvard Personal Genome Project: Lessons from participatory public research. Genome Medicine. 2014;6(2):10. (1–7) - PMC - PubMed

Publication types

Actions

MeSH terms

Actions
Actions
Actions
Actions
Actions
Actions
Actions
Actions

Grants and funding

LinkOut - more resources

Full Text Sources
Other Literature Sources
- scite Smart Citations
Miscellaneous
- NCI CPTAC Assay Portal

Save citation to file

Email citation

Add to Collections

Add to My Bibliography

Your saved search

Create a file for external citation management software

Your RSS Feed

Matching phenotypes to whole genomes: Lessons learned from four iterations of the personal genome project community challenges

Affiliations

Matching phenotypes to whole genomes: Lessons learned from four iterations of the personal genome project community challenges

Authors

Affiliations

Abstract

Figures

References

Publication types

MeSH terms

Grants and funding

LinkOut - more resources

Full Text Sources

Other Literature Sources

Miscellaneous