Online conspiracy communities are more resilient to deplatforming

Abstract

Online social media foster the creation of active communities around shared narratives. Such communities may turn into incubators for conspiracy theories-some spreading violent messages that could sharpen the debate and potentially harm society. To face these phenomena, most social media platforms implemented moderation policies, ranging from posting warning labels up to deplatforming, i.e. permanently banning users. Assessing the effectiveness of content moderation is crucial for balancing societal safety while preserving the right to free speech. In this article, we compare the shift in behavior of users affected by the ban of two large communities on Reddit, GreatAwakening and FatPeopleHate, which were dedicated to spreading the QAnon conspiracy and body-shaming individuals, respectively. Following the ban, both communities partially migrated to Voat, an unmoderated Reddit clone. We estimate how many users migrate, finding that users in the conspiracy community are much more likely to leave Reddit altogether and join Voat. Then, we quantify the behavioral shift within Reddit and across Reddit and Voat by matching common users. While in general the activity of users is lower on the new platform, GreatAwakening users who decided to completely leave Reddit maintain a similar level of activity on Voat. Toxicity strongly increases on Voat in both communities. Finally, conspiracy users migrating from Reddit tend to recreate their previous social network on Voat. Our findings suggest that banning conspiracy communities hosting violent content should be carefully designed, as these communities may be more resilient to deplatforming.

Keywords: content moderation; deplatforming; social media; social networks.

Fig. 1.

Estimated fraction of users in the four behavioral classes: Reddit remainers or leavers, and Voat joiners or nonjoiners. Users from GreatAwakening (right) are much more likely to completely leave Reddit and join Voat with respect to FatPeopleHate users (left).

Fig. 2.

Participation of users in Reddit and Voat communities. The top row shows users migrating from Reddit communities, joined up to 6 months before the ban date (on the top side of the plot) to Voat communities joined up to 6 months after the ban date (bottom side). The bottom row shows users remaining on Reddit, communities joined up to 6 months before the ban (top side of the plot) versus communities joined up to 6 months after the ban (bottom side). FatPeopleHate is on the left column and GreatAwakening on the right. In each plot, the width of each community is proportional to the number of users participating in that community, weighted by the number of messages posted there. Disclosure: This figure contains offensive language, which neither NAS nor the authors condone. Language is included as it appeared in the Reddit and Voat forums used in the research project.

Fig. 3.

Activity and toxicity on Reddit and Voat. QQ Plot comparing the distribution of the number of messages (top row) and fraction of toxic messages (bottom row) per user on GA (left column) and FPH (right column), in the 6 months before the ban on Reddit (x-axis) and 6 months after the ban on VOAT (y-axis). We distinguish between users leaving (blue dots) and remaining (green dots) on Reddit. Only Reddit leavers of the GA community maintain a level of activity comparable to the old platform, while the toxicity of the messages increases upon migration for all classes of users.

Fig. 4.

Temporal evolution of the levels of toxic speech on Reddit (x-axis) vs Voat (y-axis), on GreatAwakening (top row) and FatPeopleHate (bottom row). We distinguish between the fraction of toxic messages on Voat 6 months before the ban of the subreddit (left column), 6 months after it (middle), and up to 2 years after it (right column). The fraction of toxic comments increases in both communities on the new platform, persisting up to 2 years after the ban date.

Fig. 5.

Network overlap in the r/greatawakening (left) and r/fatpeoplehate (right) communities (vertical line), compared with a null model reshuffling the interactions (histogram). In GreatAwakening, the observed overlap is not compatible with random fluctuations. Conversely, FatPeopleHate shows an overlap fully compatible with random re-organization.

Fig. 6.

Number of new users per day in the Voat communities of v/GreatAwakening (left), v/theAwakening (middle), and v/FatPeopleHate (right) after detrending (i.e. removing the average pre-ban number of new users per day). Note that the v/theAwakening community was created the same week of the ban date, so we did not detrend the number of new users. In each plot, the first vertical dashed line indicates the date of the ban of the corresponding Reddit community and the second one corresponds to 6 months after such ban date.

Fig. 7.

QQ Plot comparing the distributions of the cosine similarity of messages authored by matched users on Reddit and Voat. We compare the similarity between matched user pairs (x-axis) and random user pairs within the same set (y-axis), for both FatPeopleHate and GreatAwakening. The similarity of messages between Reddit and Voat by matched user pairs is clearly higher than the one between random user pairs.