The h-ANN Model: Comprehensive Colonoscopy Concept Compilation Using Combined Contextual Embeddings

Abstract

Colonoscopy is a screening and diagnostic procedure for detection of colorectal carcinomas with specific quality metrics that monitor and improve adenoma detection rates. These quality metrics are stored in disparate documents i.e., colonoscopy, pathology, and radiology reports. The lack of integrated standardized documentation is impeding colorectal cancer research. Clinical concept extraction using Natural Language Processing (NLP) and Machine Learning (ML) techniques is an alternative to manual data abstraction. Contextual word embedding models such as BERT (Bidirectional Encoder Representations from Transformers) and FLAIR have enhanced performance of NLP tasks. Combining multiple clinically-trained embeddings can improve word representations and boost the performance of the clinical NLP systems. The objective of this study is to extract comprehensive clinical concepts from the consolidated colonoscopy documents using concatenated clinical embeddings. We built high-quality annotated corpora for three report types. BERT and FLAIR embeddings were trained on unlabeled colonoscopy related documents. We built a hybrid Artificial Neural Network (h-ANN) to concatenate and fine-tune BERT and FLAIR embeddings. To extract concepts of interest from three report types, 3 models were initialized from the h-ANN and fine-tuned using the annotated corpora. The models achieved best F1-scores of 91.76%, 92.25%, and 88.55% for colonoscopy, pathology, and radiology reports respectively.

Keywords: Clinical Concept Extraction; Colonoscopy; Deep Learning; Natural Language Processing; Word Embeddings.

Figure 1:

Colonoscopy taxonomy depicting clinical entities and their classifications. Colonoscopy reports were annotated for entities mentioned in the taxonomy.

Figure 2:

Pathology taxonomy depicting clinical entities and their classifications. Pathology reports were annotated for entities mentioned in the taxonomy.

Figure 3:

Radiology imaging taxonomy depicting clinical entities and their classifications. Radiology reports were annotated for entities mentioned in the taxonomy.

Figure 4:

Workflow depicting training of language models, concatenating embeddings, instantiating and fine-tuning h-ANN models to extract clinical concepts from colonoscopy related documents. GI: Gastroenterology, h-ANN: Hybrid artificial neural network.

Figure 5:

The h-ANN architecture depicting embedding, Bi-LSTM, and CRF layers. Concatenated BERT and FLAIR embeddings are given as input features to the Bi-LSTM layer. BERT: Bidirectional Encoder Representations from Transformers.

Figure 6:

Training curves for h-ANNpath, h-ANNcol, and h-ANNrad models. F1 score on the test set was measured after completion of each epochs.