Learning adaptive representations for entity recognition in the biomedical domain

Abstract

Background: Named Entity Recognition is a common task in Natural Language Processing applications, whose purpose is to recognize named entities in textual documents. Several systems exist to solve this task in the biomedical domain, based on Natural Language Processing techniques and Machine Learning algorithms. A crucial step of these applications is the choice of the representation which describes data. Several representations have been proposed in the literature, some of which are based on a strong knowledge of the domain, and they consist of features manually defined by domain experts. Usually, these representations describe the problem well, but they require a lot of human effort and annotated data. On the other hand, general-purpose representations like word-embeddings do not require human domain knowledge, but they could be too general for a specific task.

Results: This paper investigates methods to learn the best representation from data directly, by combining several knowledge-based representations and word embeddings. Two mechanisms have been considered to perform the combination, which are neural networks and Multiple Kernel Learning. To this end, we use a hybrid architecture for biomedical entity recognition which integrates dictionary look-up (also known as gazetteers) with machine learning techniques. Results on the CRAFT corpus clearly show the benefits of the proposed algorithm in terms of F₁ score.

Conclusions: Our experiments show that the principled combination of general, domain specific, word-, and character-level representations improves the performance of entity recognition. We also discussed the contribution of each representation in the final solution.

Keywords: Ensemble; Kernel methods; Named entity recognition; Neural networks.

Fig. 1

A depiction of a simple neural network. Arrows define the non-linear transformations from the input (green) to the output layer (red). Hidden layers are described in blue. Circles denote atomic features (or neurons)

Fig. 2

A Neural Network architecture to combine and integrate different sources and representations. Each base network is trained and validated on a single base representation. At the top of the network, a shared layer combines the output of the base networks. Green circles denote the input features. In the example, the network combines the output from P different neural networks, i.e. P different representations

Fig. 3

A depiction of the hybrid BNER system. A term is retrieved if it is selected by the dictionary look-up and it is accepted by the ML algorithm

Fig. 4

Depiction of the proposed system and other features aggregation schemes. The OGER annotator retrieves candidate entities from input texts (Deoxyribonucleic acid in the example). Then, different set of features associate to the candidate entity are computed (e.g. affixes and spectrum) or extracted (word2vec), producing multiple feature vectors. Consequently, the features aggregation schema defines the final representation as (i) a single base representation, (ii) the concatenation of base feature vectors, and (iii) the principled combination obtained through a MKL algorithm or a NN (shown in Fig. 2). The resulting representation is used with a classifier to select the final class (entity or not)

Fig. 5

Contribution of base representations in the BioNER