Skip to main page content
U.S. flag

An official website of the United States government

Dot gov

The .gov means it’s official.
Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you’re on a federal government site.

Https

The site is secure.
The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

Access keys NCBI Homepage MyNCBI Homepage Main Content Main Navigation
. 2024 Mar 15;14(1):6320.
doi: 10.1038/s41598-024-56976-5.

Application of the transformer model algorithm in chinese word sense disambiguation: a case study in chinese language

Linlin Li  1 , Juxing Li  2 , Hongli Wang  3 , Jianing Nie  4
Affiliations

Application of the transformer model algorithm in chinese word sense disambiguation: a case study in chinese language

Linlin Li et al. Sci Rep. .

Abstract

This study aims to explore the research methodology of applying the Transformer model algorithm to Chinese word sense disambiguation, seeking to resolve word sense ambiguity in the Chinese language. The study introduces deep learning and designs a Chinese word sense disambiguation model based on the fusion of the Transformer with the Bi-directional Long Short-Term Memory (BiLSTM) algorithm. By utilizing the self-attention mechanism of Transformer and the sequence modeling capability of BiLSTM, this model efficiently captures semantic information and context relationships in Chinese sentences, leading to accurate word sense disambiguation. The model's evaluation is conducted using the PKU Paraphrase Bank, a Chinese text paraphrase dataset. The results demonstrate that the model achieves a precision rate of 83.71% in Chinese word sense disambiguation, significantly outperforming the Long Short-Term Memory algorithm. Additionally, the root mean squared error of this algorithm is less than 17, with a loss function value remaining around 0.14. Thus, this study validates that the constructed Transformer-fused BiLSTM-based Chinese word sense disambiguation model algorithm exhibits both high accuracy and robustness in identifying word senses in the Chinese language. The findings of this study provide valuable insights for advancing the intelligent development of word senses in Chinese language applications.

Keywords: BiLSTM; Chinese language; Root mean squared error; Transformer model algorithm; Word sense disambiguation.

PubMed Disclaimer

Conflict of interest statement

The authors declare no competing interests.

Figures

Figure 1
Figure 1
Data preprocessing flowchart.
Figure 2
Figure 2
Application of the Transformer model framework in NLP.
Algorithm 1
Algorithm 1
Pseudocode for applying the Transformer model to Chinese word sense disambiguation
Figure 3
Figure 3
Flow of BiLSTM applied to NLP.
Figure 4
Figure 4
Framework of the Transformer-fused BiLSTM-based Chinese word sense disambiguation model.
Figure 5
Figure 5
Loss function results.
Figure 6
Figure 6
Accuracy results of each algorithm in the training set.
Figure 7
Figure 7
Accuracy results of each algorithm in the test set.
Figure 8
Figure 8
RMSE results of each algorithm in the training set under different sentence pairs.
Figure 9
Figure 9
RMSE results of each algorithm in the test set under different sentence pairs.
See this image and copyright information in PMC

Similar articles

See all similar articles

Cited by

References

    1. Bharadiya J. A comprehensive survey of deep learning techniques natural language processing[J] Eur. J. Technol. 2023;7(1):58–66. doi: 10.47672/ejt.1473. - DOI
    1. Xu N, Ma L, Wang L, et al. Extracting domain knowledge elements of construction safety management: Rule-based approach using Chinese natural language processing[J] J. Manag. Eng. 2021;37(2):04021001. doi: 10.1061/(ASCE)ME.1943-5479.0000870. - DOI
    1. Zheng Z, Lu XZ, Chen KY, et al. Pretrained domain-specific language model for natural language processing tasks in the AEC domain[J] Comput. Ind. 2022;142:103733. doi: 10.1016/j.compind.2022.103733. - DOI
    1. Pan DJ, Lin D. Cognitive–linguistic skills explain Chinese reading comprehension within and beyond the simple view of reading in Hong Kong kindergarteners[J] Lang. Learn. 2023;73(1):126–160. doi: 10.1111/lang.12515. - DOI
    1. Guo H. Chinese primary school students’ translanguage in EFL classrooms: What is it and why is it needed? [J] Asia-Pacific Educ. Res. 2023;32(2):211–226.