Dutch tweets before, during, and after a black lives matter demonstration in Amsterdam: Expert annotated data, protocol, and labelling tool

Abstract

The Netherlands police are looking for measures to examine sentiment on social media related to protest demonstrations. While models exist to detect more subtle expressions of sentiment within tweets, models trained in the Dutch language are scarce. Being able to predict sentiment development during protests is relevant for parties like the Dutch government and the police to get more insight to when and where potential law enforcement is needed for public order and safety. Therefore, to analyse sentiment before, during, and after protest demonstrations, data was collected with tweets related to a Black Lives Matter protest that took place in Amsterdam during the COVID-19 pandemic. All tweets have been manually labelled by a dedicated open-source intelligence (OSINT) team within the Netherlands police following an established protocol. Both the data and the protocol are available, and interesting for researchers in natural language processing, topic detection, sentiment analysis, and protests analysis. The developed labelling tool for the labelling process is publicly available.

Keywords: COVID-19; Manual labelling; Natural language processing; Protest; Social media analysis.

Fig. 1

This figure represents the ratio of label counts per labeller. The labeller names are hashed. The leftmost column represents the ratio of all labels. Note that the number of labelled tweets differs between labellers, see Table 5.

Fig. 2

Representation of the Tweeti labelling tool used to label the majority of the tweets. The figure shows the label categories (Relevance, Incident Related, and Behavioural Relevance), the labels (Mainstream Media, Verified Source, and Other Source), a selected label (Related Topic), and the function buttons (Previous Tweet, Next Tweet, Exclude Tweet, Discuss Tweet). Note that the content of the displayed tweet is covered by a grey square.