Identifying relations of medications with adverse drug events using recurrent convolutional neural networks and gradient boosting

Abstract

Objective: To develop a natural language processing system that identifies relations of medications with adverse drug events from clinical narratives. This project is part of the 2018 n2c2 challenge.

Materials and methods: We developed a novel clinical named entity recognition method based on an recurrent convolutional neural network and compared it to a recurrent neural network implemented using the long-short term memory architecture, explored methods to integrate medical knowledge as embedding layers in neural networks, and investigated 3 machine learning models, including support vector machines, random forests and gradient boosting for relation classification. The performance of our system was evaluated using annotated data and scripts provided by the 2018 n2c2 organizers.

Results: Our system was among the top ranked. Our best model submitted during this challenge (based on recurrent neural networks and support vector machines) achieved lenient F1 scores of 0.9287 for concept extraction (ranked third), 0.9459 for relation classification (ranked fourth), and 0.8778 for the end-to-end relation extraction (ranked second). We developed a novel named entity recognition model based on a recurrent convolutional neural network and further investigated gradient boosting for relation classification. The new methods improved the lenient F1 scores of the 3 subtasks to 0.9292, 0.9633, and 0.8880, respectively, which are comparable to the best performance reported in this challenge.

Conclusion: This study demonstrated the feasibility of using machine learning methods to extract the relations of medications with adverse drug events from clinical narratives.

Keywords: clinical natural language processing; deep learning; named entity recognition; recurrent convolutional neural network; relation extraction.

Figure 1.

Main architecture of the long short term-memory (LSTM) with a CRFs layer (ie, the LSTM-CRFs) model.

Figure 2.

Main architecture of the recurrent convolutional neural network (RCNN) model.

Figure 3.

Performance of SVMs when considering candidate relations with cross-distances ≤ N. In SVMs, N denotes the SVMs model that considered relations with a cross-distance less or equal than N. For example, SVMs-2 contains 3 classifiers handling relations with cross-distance in [0, 1, 2].