Investigating toxicity changes of cross-community redditors from 2 billion posts and comments

Abstract

This research investigates changes in online behavior of users who publish in multiple communities on Reddit by measuring their toxicity at two levels. With the aid of crowdsourcing, we built a labeled dataset of 10,083 Reddit comments, then used the dataset to train and fine-tune a Bidirectional Encoder Representations from Transformers (BERT) neural network model. The model predicted the toxicity levels of 87,376,912 posts from 577,835 users and 2,205,581,786 comments from 890,913 users on Reddit over 16 years, from 2005 to 2020. This study utilized the toxicity levels of user content to identify toxicity changes by the user within the same community, across multiple communities, and over time. As for the toxicity detection performance, the BERT model achieved a 91.27% classification accuracy and an area under the receiver operating characteristic curve (AUC) score of 0.963 and outperformed several baseline machine learning and neural network models. The user behavior toxicity analysis showed that 16.11% of users publish toxic posts, and 13.28% of users publish toxic comments. However, results showed that 30.68% of users publishing posts and 81.67% of users publishing comments exhibit changes in their toxicity across different communities, indicating that users adapt their behavior to the communities' norms. Furthermore, time series analysis with the Granger causality test of the volume of links and toxicity in user content showed that toxic comments are Granger caused by links in comments.

Keywords: Machine learning; Online communities; Online hate; Posting behavior; Reddit; Toxicity.

Figure 1. A Reddit post from the subreddit “r/science” with its associated discussion threads.

Figure 2. Cumulative distribution function of the participating subreddits per user in (A) posts and (B) comments.

Figure 3. The count of toxicity changes over time in posting users from condition 1 (NT →T), condition 2 (T →NT), and both conditions combined.

Figure 4. The count of toxicity changes over time in commenting users from condition 1 (NT →T), condition 2 (T →NT), and both conditions combined.

Figure 5. (A–D) Heatmap plots of the Δ in user posts and comments over two pairs of years.

The dark color in the heatmap plot denotes scattered deltas while the light colors denotes concentrated deltas in specific locations.

Figure 6. The total amount of Δ in posting users content over time with an interpolation of Δ averages across three year intervals.

Figure 7. The total amount of Δ in commenting users content over time with an interpolation of Δ averages across three year intervals.

Figure 8. The total number of posts, toxic posts, and links in every year followed by the normalized totals using the min-max scale.

Figure 9. The distribution of internal and external links, followed by the total number of known media types and image links from the posts collection.

Figure 10. The total number of comments, toxic comments, and links in every year followed by the normalized totals using the min-max scale.

Figure 11. The distribution of internal and external links, followed by the total number of known media types and image links from the comments collection.

Figure 12. Correlation between the total number of participating subreddits over time and (A) the total number of toxic posts and (B) the total number of toxic comments.

Figure B1. The labeling task instructions that we provided to crowd workers.

Figure B2. The validation questions that crowd workers had to pass before beginning labeling.