A Natural Language Processing (NLP) Evaluation on COVID-19 Rumour Dataset Using Deep Learning Techniques

Abstract

Context and Background: Since December 2019, the coronavirus (COVID-19) epidemic has sparked considerable alarm among the general community and significantly affected societal attitudes and perceptions. Apart from the disease itself, many people suffer from anxiety and depression due to the disease and the present threat of an outbreak. Due to the fast propagation of the virus and misleading/fake information, the issues of public discourse alter, resulting in significant confusion in certain places. Rumours are unproven facts or stories that propagate and promote sentiments of prejudice, hatred, and fear. Objective. The study's objective is to propose a novel solution to detect fake news using state-of-the-art machines and deep learning models. Furthermore, to analyse which models outperformed in detecting the fake news. Method. In the research study, we adapted a COVID-19 rumours dataset, which incorporates rumours from news websites and tweets, together with information about the rumours. It is important to analyse data utilizing Natural Language Processing (NLP) and Deep Learning (DL) approaches. Based on the accuracy, precision, recall, and the f1 score, we can assess the effectiveness of the ML and DL algorithms. Results. The data adopted from the source (mentioned in the paper) have collected 9200 comments from Google and 34,779 Twitter postings filtered for phrases connected with COVID-19-related fake news. Experiment 1. The dataset was assessed using the following three criteria: veracity, stance, and sentiment. In these terms, we have different labels, and we have applied the DL algorithms separately to each term. We have used different models in the experiment such as (i) LSTM and (ii) Temporal Convolution Networks (TCN). The TCN model has more performance on each measurement parameter in the evaluated results. So, we have used the TCN model for the practical implication for better findings. Experiment 2. In the second experiment, we have used different state-of-the-art deep learning models and algorithms such as (i) Simple RNN; (ii) LSTM + Word Embedding; (iii) Bidirectional + Word Embedding; (iv) LSTM + CNN-1D; and (v) BERT. Furthermore, we have evaluated the performance of these models on all three datasets, e.g., veracity, stance, and sentiment. Based on our second experimental evaluation, the BERT has a superior performance over the other models compared.

Figure 1

Examples of data structure adopted from [24].

Figure 2

COVID-19 rumour identification using AI techniques.

Figure 3

Flowchart depicting the collecting, labelling, and postprocessing of datasets.

Figure 4

Labels in the dataset are highlighted—adopted from [24].

Figure 5

Schematic of the long short-term memory cell—adopted from [47].

Figure 6

Schematic of the Temporal Convolutional Networks—adopted from [50].

Figure 7

LSTM—the graphical representation of evaluating results for sentiment dataset.

Figure 8

TCN—the graphical representation of evaluating results for sentiment dataset.

Figure 9

LSTM—the graphical representation of evaluating results for veracity dataset.

Figure 10

TCN—the graphical representation of evaluating results for veracity dataset.

Figure 11

LSTM—the graphical representation of evaluating results for stance dataset.

Figure 12

TCN—the graphical representation of evaluating results for stance dataset.

Figure 13

Summary of simple RNN model.

Figure 14

Summary of LSTM + Word Embedding model.

Figure 15

Summary of Bidirectional + Word Embedding model.

Figure 16

Summary of LSTM + CNN-1D model.