BioADAPT-MRC: adversarial learning-based domain adaptation improves biomedical machine reading comprehension task

Abstract

Motivation: Biomedical machine reading comprehension (biomedical-MRC) aims to comprehend complex biomedical narratives and assist healthcare professionals in retrieving information from them. The high performance of modern neural network-based MRC systems depends on high-quality, large-scale, human-annotated training datasets. In the biomedical domain, a crucial challenge in creating such datasets is the requirement for domain knowledge, inducing the scarcity of labeled data and the need for transfer learning from the labeled general-purpose (source) domain to the biomedical (target) domain. However, there is a discrepancy in marginal distributions between the general-purpose and biomedical domains due to the variances in topics. Therefore, direct-transferring of learned representations from a model trained on a general-purpose domain to the biomedical domain can hurt the model's performance.

Results: We present an adversarial learning-based domain adaptation framework for the biomedical machine reading comprehension task (BioADAPT-MRC), a neural network-based method to address the discrepancies in the marginal distributions between the general and biomedical domain datasets. BioADAPT-MRC relaxes the need for generating pseudo labels for training a well-performing biomedical-MRC model. We extensively evaluate the performance of BioADAPT-MRC by comparing it with the best existing methods on three widely used benchmark biomedical-MRC datasets-BioASQ-7b, BioASQ-8b and BioASQ-9b. Our results suggest that without using any synthetic or human-annotated data from the biomedical domain, BioADAPT-MRC can achieve state-of-the-art performance on these datasets.

Availability and implementation: BioADAPT-MRC is freely available as an open-source project at https://github.com/mmahbub/BioADAPT-MRC.

Supplementary information: Supplementary data are available at Bioinformatics online.

Fig. 1.

BioADAPT-MRC: an BioADAPT-MRC task. The framework has three main components: (i) feature extractor M_F, (ii) MRC-module M_Q and (iii) domain-similarity discriminator D

Fig. 2.

Per-epoch cosine distance between source-domain training sample pairs and source- and target-domain training sample pairs across 50 epochs

Fig. 3.

Mean accuracy scores (left) and mean silhouette scores (right) with standard deviations for DBSCAN clustering on BioASQ test sets and SQuAD

Fig. 4.

Error analysis of BioADAPT-MRC, in comparison with the baseline model, depending on the types of questions in the BioASQ test sets

Fig. 5.

Error analysis of BioADAPT-MRC, in comparison with the baseline model, depending on the types of answers in the BioASQ test sets

Fig. 6.

Example question–answer pairs from the test sets demonstrating the strengths and weaknesses of the BioADAPT-MRC model over the baseline model. The green and red colors show correctly and incorrectly predicted answers, respectively (A color version of this figure appears in the online version of this article.)