PubMedPortable: A Framework for Supporting the Development of Text Mining Applications

Abstract

Information extraction from biomedical literature is continuously growing in scope and importance. Many tools exist that perform named entity recognition, e.g. of proteins, chemical compounds, and diseases. Furthermore, several approaches deal with the extraction of relations between identified entities. The BioCreative community supports these developments with yearly open challenges, which led to a standardised XML text annotation format called BioC. PubMed provides access to the largest open biomedical literature repository, but there is no unified way of connecting its data to natural language processing tools. Therefore, an appropriate data environment is needed as a basis to combine different software solutions and to develop customised text mining applications. PubMedPortable builds a relational database and a full text index on PubMed citations. It can be applied either to the complete PubMed data set or an arbitrary subset of downloaded PubMed XML files. The software provides the infrastructure to combine stand-alone applications by exporting different data formats, e.g. BioC. The presented workflows show how to use PubMedPortable to retrieve, store, and analyse a disease-specific data set. The provided use cases are well documented in the PubMedPortable wiki. The open-source software library is small, easy to use, and scalable to the user's system requirements. It is freely available for Linux on the web at https://github.com/KerstenDoering/PubMedPortable and for other operating systems as a virtual container. The approach was tested extensively and applied successfully in several projects.

Fig 1. PubMedPortable workflow.

1) Download XML files from PubMed. 2) Parse and upload data into a PostgreSQL relational database. 3) Build a Xapian full text index. 4) Develop text mining applications.

Fig 2. General BioC workflow.

This is the minimalistic approach from Comeau et al. [10] with the example how to add MeSH terms to BioC PubMed titles and abstracts from the PubMedPortable PostgreSQL database.

Fig 3. Excerpt of a BioC XML document.

The document ID 100475 is a PubMed ID. PubTator annotations are shown with infon elements that contain the key type with the value Disease and the key MEDIC referring to a MeSH ID, such as D010190 for the given disease pancreatic carcinoma. The PubMedPortable MeSH term annotations are shown with the annotation IDs 0_MeSH and 1_MeSH to make them distinguishable from the normally iterating PubTator annotation IDs. They were added after calling the PubTator web service.

Fig 4. Read BioC elements.

All BioC XML elements can be read with the BioC API. The script refers to the left part of the workflow shown in Fig 2. Iterating over the given annotations as shown in Fig 3 will e.g. show Annotation ID: 0, Annotation Type: Disease, Annotation Text: pancreatic carcinoma, and Offset and term length: 77:20.

Fig 5. Documentation to generate a word cloud using PubMedPortable.

Different tools and different data formats might be used for named-entity recognition. Tab-separated files (CSV) with PubMed ID, synonym, and identifier in each line are used to collect all abstracts in which a match for identifier-specific synonyms appeared.

Fig 6. Genes, proteins, chemicals, and diseases related to pancreatic cancer.

The 150 most frequently appearing entities in terms of their number of abstracts were identified with DNorm [28], GeneTUKit [27], and PubTator [16]. Fig 5 shows the steps to generate this word cloud.

Fig 7. Timelines for the publications of the genes KRAS, BRCA2, and CDKN2A until 2014.

The PubMed IDs for these three genes were extracted from the list of entities resulting from step 4 in Fig 5. The publication years were selected from the PubMedPortable database.

Fig 8. Boolean query result.

The HTML page shows a rank in the first column with a relative match score, scaled to 100. The NEAR condition was used to allow up to four other words between the drug erlotinib and the disease term pancreatic cancer without fixed word order.