Skip to main page content
U.S. flag

An official website of the United States government

Dot gov

The .gov means it’s official.
Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you’re on a federal government site.

Https

The site is secure.
The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

Access keys NCBI Homepage MyNCBI Homepage Main Content Main Navigation
. 2005;6 Suppl 1(Suppl 1):S1.
doi: 10.1186/1471-2105-6-S1-S1. Epub 2005 May 24.

Overview of BioCreAtIvE: critical assessment of information extraction for biology

Lynette Hirschman  1 , Alexander Yeh, Christian Blaschke, Alfonso Valencia
Affiliations

Overview of BioCreAtIvE: critical assessment of information extraction for biology

Lynette Hirschman et al. BMC Bioinformatics. 2005.

Abstract

Background: The goal of the first BioCreAtIvE challenge (Critical Assessment of Information Extraction in Biology) was to provide a set of common evaluation tasks to assess the state of the art for text mining applied to biological problems. The results were presented in a workshop held in Granada, Spain March 28-31, 2004. The articles collected in this BMC Bioinformatics supplement entitled "A critical assessment of text mining methods in molecular biology" describe the BioCreAtIvE tasks, systems, results and their independent evaluation.

Results: BioCreAtIvE focused on two tasks. The first dealt with extraction of gene or protein names from text, and their mapping into standardized gene identifiers for three model organism databases (fly, mouse, yeast). The second task addressed issues of functional annotation, requiring systems to identify specific text passages that supported Gene Ontology annotations for specific proteins, given full text articles.

Conclusion: The first BioCreAtIvE assessment achieved a high level of international participation (27 groups from 10 countries). The assessment provided state-of-the-art performance results for a basic task (gene name finding and normalization), where the best systems achieved a balanced 80% precision / recall or better, which potentially makes them suitable for real applications in biology. The results for the advanced task (functional annotation from free text) were significantly lower, demonstrating the current limitations of text-mining approaches where knowledge extrapolation and interpretation are required. In addition, an important contribution of BioCreAtIvE has been the creation and release of training and test data sets for both tasks. There are 22 articles in this special issue, including six that provide analyses of results or data quality for the data sets, including a novel inter-annotator consistency assessment for the test set used in task 2.

PubMed Disclaimer

Figures

Figure 1
Figure 1
Curation pipeline for a model organism database.
Figure 2
Figure 2
Sample abstract with unique gene identifiers, plus excerpt from lexicon.
Figure 3
Figure 3
SWISS-PROT ID, GO code, PubMed ID triple and the corresponding evidence passage from the article (3 examples).
See this image and copyright information in PMC

References

    1. Yeh AS, Morgan A, Colosimo M, Hirschman L. BioCreAtIvE task 1A: gene mention finding evaluation. BMC Bioinformatics. 2005;6:S2. doi: 10.1186/1471-2105-6-S1-S2. - DOI - PMC - PubMed
    1. Tanabe L, Xie N, Thom LH, Matten W, Wilbur WJ. GENETAG: A Tagged Corpus for Gene/Protein Named Entity Recognition. BMC Bioinformatics. 2005;6:S3. doi: 10.1186/1471-2105-6-S1-S3. - DOI - PMC - PubMed
    1. Kinoshita S, Cohen KB, Ogren PV, Hunter L. BioCreAtIvE Task 1A: Entity Identification with a Stochastic Tagger. BMC Bioinformatics. 2005;6:S4. doi: 10.1186/1471-2105-6-S1-S4. - DOI - PMC - PubMed
    1. Finkel J, Dingare S, Manning CD, Nissim M, Alex B, Grover C. Exploring the Boundaries: Gene and Protein Identification in Biomedical Text. BMC Bioinformatics. 2005;6:S5. doi: 10.1186/1471-2105-6-S1-S5. - DOI - PMC - PubMed
    1. McDonald R, Pereira F. Identifying Gene and Protein Mentions in Text using Conditional Random Fields. BMC Bioinformatics. 2005;6:S6. doi: 10.1186/1471-2105-6-S1-S6. - DOI - PMC - PubMed

Publication types