Tracking mosquito-borne diseases via social media: a machine learning approach to topic modelling and sentiment analysis

Abstract

Mosquito-borne diseases (MBDs) are a major threat worldwide, and public consultation on these diseases is critical to disease control decision-making. However, traditional public surveys are time-consuming and labor-intensive and do not allow for timely decision-making. Recent studies have explored text analytic approaches to elicit public comments from social media for public health. Therefore, this study aims to demonstrate a text analytics pipeline to identify the MBD topics that were discussed on Twitter and significantly influenced public opinion. A total of 25,000 tweets were retrieved from Twitter, topics were modelled using LDA and sentiment polarities were calculated using the VADER model. After data cleaning, we obtained a total of 6,243 tweets, which we were able to process with the feature selection algorithms. Boruta was used as a feature selection algorithm to determine the importance of topics to public opinion. The result was validated using multinomial logistic regression (MLR) performance and expert judgement. Important issues such as breeding sites, mosquito control, impact/funding, time of year, other diseases with similar symptoms, mosquito-human interaction and biomarkers for diagnosis were identified by both LDA and experts. The MLR result shows that the topics selected by LASSO perform significantly better than the other algorithms, and the experts further justify the topics in the discussion.

Keywords: Amplification intelligence; Determinants; Infection diseases; Mosquitoes; Text mining.

Figure 1. Text analytics pipeline and the role of machine and human, respectively.

Figure 2. Figure 2 shows the overall workflow to construct the annotated dataset.

Figure 3. Distribution of tweets by keyword and the corresponding word cloud.

The size of the word indicates the density of tweets mentioning the word, e.g., the 2,435 tweets queried with “dengue” mainly mentioned “mosquito”, “fever”, “health” and ”malaria”, as indicated by the larger front size of the word in the word cloud.

Figure 4. Distribution of sentiment polarity of tweets in this study.

Figure 5. Venn diagram of the topics selected by feature selection algorithm.

Figure 6. Performance of multinomial logistics regression by using different selected topic group.

Mean ± standard error as the confidence interval.

Figure 7. The distribution of polarity for the individual topic.

Three topics with a majority of negative polarity indicated by a larger grey area: “Symptom/outcome of an outbreak”, “Diseases with similar symptoms”. Two topics with a majority positive polarity, indicated by a larger blue area in the pie chart, were “Efforts/funding” and “Biomarkers for diagnosis”. Two topics with neutral polarity, indicated by a larger orange area in the pie chart, were “Mosquito control” and “Season”.