Skip to main page content
U.S. flag

An official website of the United States government

Dot gov

The .gov means it’s official.
Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you’re on a federal government site.

Https

The site is secure.
The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

Access keys NCBI Homepage MyNCBI Homepage Main Content Main Navigation
. 2024 Apr 26;14(1):9603.
doi: 10.1038/s41598-024-60210-7.

A multimodal approach to cross-lingual sentiment analysis with ensemble of transformer and LLM

Md Saef Ullah Miah  1 , Md Mohsin Kabir  2 , Talha Bin Sarwar  1 , Mejdl Safran  3 , Sultan Alfarhood  4 , M F Mridha  5
Affiliations

A multimodal approach to cross-lingual sentiment analysis with ensemble of transformer and LLM

Md Saef Ullah Miah et al. Sci Rep. .

Abstract

Sentiment analysis is an essential task in natural language processing that involves identifying a text's polarity, whether it expresses positive, negative, or neutral sentiments. With the growth of social media and the Internet, sentiment analysis has become increasingly important in various fields, such as marketing, politics, and customer service. However, sentiment analysis becomes challenging when dealing with foreign languages, particularly without labelled data for training models. In this study, we propose an ensemble model of transformers and a large language model (LLM) that leverages sentiment analysis of foreign languages by translating them into a base language, English. We used four languages, Arabic, Chinese, French, and Italian, and translated them using two neural machine translation models: LibreTranslate and Google Translate. Sentences were then analyzed for sentiment using an ensemble of pre-trained sentiment analysis models: Twitter-Roberta-Base-Sentiment-Latest, bert-base-multilingual-uncased-sentiment, and GPT-3, which is an LLM from OpenAI. Our experimental results showed that the accuracy of sentiment analysis on translated sentences was over 86% using the proposed model, indicating that foreign language sentiment analysis is possible through translation to English, and the proposed ensemble model works better than the independent pre-trained models and LLM.

Keywords: Cross-lingual communication; Ensemble with LLM; Neural machine translation; Pretrained sentiment analyzer model; Sentiment analysis.

PubMed Disclaimer

Conflict of interest statement

The authors declare no competing interests.

Figures

Figure 1
Figure 1
Overview of the proposed method.
Figure 2
Figure 2
Distribution of different languages in the dataset used in this study.
Algorithm 1
Algorithm 1
Text Cleaner Algorithm
Algorithm 2
Algorithm 2
Translation Process
Algorithm 3
Algorithm 3
Sentiment Analysis Ensemble Model
Algorithm 4
Algorithm 4
Defining TP, TN, FP and FN
Figure 3
Figure 3
Experimental results showing the outcomes of different evaluation metrics.
Figure 4
Figure 4
Confusion matrices from different experiments.
See this image and copyright information in PMC

References

    1. Yadav A, Vishwakarma DK. Sentiment analysis using deep learning architectures: A review. Artif. Intell. Rev. 2020;53:4335–4385. doi: 10.1007/s10462-019-09794-5. - DOI
    1. Gandhi A, Adhvaryu K, Poria S, Cambria E, Hussain A. Multimodal sentiment analysis: A systematic review of history, datasets, multimodal fusion methods, applications, challenges and future directions. Inf. Fusion. 2023;91:424–444. doi: 10.1016/j.inffus.2022.09.025. - DOI
    1. Cambria, E., Das, D., Bandyopadhyay, S. & Feraco, A. Affective computing and sentiment analysis. In A Practical Guide to Sentiment Analysis 1–10 (2017).
    1. Sarker IH. Machine learning: Algorithms, real-world applications and research directions. SN Comput. Sci. 2021;2:160. doi: 10.1007/s42979-021-00592-x. - DOI - PMC - PubMed
    1. Das, R. & Singh, T. D. Multimodal sentiment analysis: A survey of methods, trends and challenges. ACM Comput. Surv. (2023).