Leveraging machine learning-based approaches to assess human papillomavirus vaccination sentiment trends with Twitter data

Abstract

Background: As one of the serious public health issues, vaccination refusal has been attracting more and more attention, especially for newly approved human papillomavirus (HPV) vaccines. Understanding public opinion towards HPV vaccines, especially concerns on social media, is of significant importance for HPV vaccination promotion.

Methods: In this study, we leveraged a hierarchical machine learning based sentiment analysis system to extract public opinions towards HPV vaccines from Twitter. English tweets containing HPV vaccines-related keywords were collected from November 2, 2015 to March 28, 2016. Manual annotation was done to evaluate the performance of the system on the unannotated tweets corpus. Followed time series analysis was applied to this corpus to track the trends of machine-deduced sentiments and their associations with different days of the week.

Results: The evaluation of the unannotated tweets corpus showed that the micro-averaging F scores have reached 0.786. The learning system deduced the sentiment labels for 184,214 tweets in the collected unannotated tweets corpus. Time series analysis identified a coincidence between mainstream outcome and Twitter contents. A weak trend was found for "Negative" tweets that decreased firstly and began to increase later; an opposite trend was identified for "Positive" tweets. Tweets that contain the worries on efficacy for HPV vaccines showed a relative significant decreasing trend. Strong associations were found between some sentiments ("Positive", "Negative", "Negative-Safety" and "Negative-Others") with different days of the week.

Conclusions: Our efforts on sentiment analysis for newly approved HPV vaccines provide us an automatic and instant way to extract public opinion and understand the concerns on Twitter. Our approaches can provide a feedback to public health professionals to monitor online public response, examine the effectiveness of their HPV vaccination promotion strategies and adjust their promotion plans.

Keywords: Hierarchical classification; Human papillomavirus vaccines; Machine learning; Sentiment analysis; Twitter.

Fig. 1

Sentiment classification scheme for HPV vaccine related tweets [12]

Fig. 2

Overview of the machine learning based system for tweets sentiment analysis

Fig. 3

Sentiments distribution in large scale unannotated HPV vaccines related tweets corpus. (Neg: Negative)

Fig. 4

Stacked line chart for the number of tweets containing different sentiments from November 2, 2015 to March 28, 2016

Fig. 5

The relative proportions of tweets containing Negative, Neutral and Positive opinions

Fig. 6

The relative proportions of “NegSafety”, “NegEfficacy” and “NegOthers” tweets to “Negative” tweets

Fig. 7

The association of different days of the week with the relative proportions of tweets containing Negative, Neutral and Positive opinions

Fig. 8

The association of different days of the week with the relative proportions of “NegSafety”, “NegEfficacy” and “NegOthers” tweets