Impact analysis of keyword extraction using contextual word embedding

Abstract

A document's keywords provide high-level descriptions of the content that summarize the document's central themes, concepts, ideas, or arguments. These descriptive phrases make it easier for algorithms to find relevant information quickly and efficiently. It plays a vital role in document processing, such as indexing, classification, clustering, and summarization. Traditional keyword extraction approaches rely on statistical distributions of key terms in a document for the most part. According to contemporary technological breakthroughs, contextual information is critical in deciding the semantics of the work at hand. Similarly, context-based features may be beneficial in the job of keyword extraction. For example, simply indicating the previous or next word of the phrase of interest might be used to describe the context of a phrase. This research presents several experiments to validate that context-based key extraction is significant compared to traditional methods. Additionally, the KeyBERT proposed methodology also results in improved results. The proposed work relies on identifying a group of important words or phrases from the document's content that can reflect the authors' main ideas, concepts, or arguments. It also uses contextual word embedding to extract keywords. Finally, the findings are compared to those obtained using older approaches such as Text Rank, Rake, Gensim, Yake, and TF-IDF. The Journals of Universal Computer (JUCS) dataset was employed in our research. Only data from abstracts were used to produce keywords for the research article, and the KeyBERT model outperformed traditional approaches in producing similar keywords to the authors' provided keywords. The average similarity of our approach with author-assigned keywords is 51%.

Keywords: Contextual Word Embedding; Keyword extraction; TF-IDF; Text Rank; Yake.

Figure 1. Flow diagram of the proposed approach.

Figure 2. Contextual word embedding by BERT.

Figure 3. Comparison of each document similarity with author-assigned keywords in different approaches.

Figure 4. Keyword extraction analysis with different approaches.

Figure 5. Comparison of each document similarity with author-assigned keywords in different approaches using Wordnet synonyms.

Figure 6. Keyword extraction analysis with different approaches using wordnet synonym.

Figure 7. Comparison of each document similarity with Author assigned keywords in different approaches using Wordnet synonyms.

Figure 8. Keyword extraction analysis with different approaches using Wordnet synonym.