Engagement with fact-checked posts on Reddit

Abstract

Contested factual claims shared online are of increasing interest to scholars and the public. Characterizing temporal patterns of sharing and engagement with such information, as well as the effect of sharing associated fact-checks, can help us understand the online political news environment more fully. Here, we investigate differential engagement with fact-checked posts shared online via Reddit from 2016 to 2018. The data comprise ∼29,000 conversations, ∼849,000 users, and ∼9.8 million comments. We classified the veracity of the posts being discussed as true, mixed, or false using three fact-checking organizations. Regardless of veracity, fact-checked posts had larger and longer lasting conversations than claims that were not fact-checked. Among those that were fact-checked, posts rated as false were discussed less and for shorter periods of time than claims that were rated as true. We also observe that fact-checks of posts rated as false tend to happen more quickly than fact-checks of posts rated as true. Finally, we observe that thread deletion and removal are systematically related to the presence of a fact-check and the veracity of the fact-check, but when deletion and removal are combined the differences are minimal. Theoretical and practical implications of the findings are discussed.

Keywords: contested news; engagement; fact-checks; social media.

Fig. 1.

Thread structure, most frequent subreddits, and rates of threads over time. Panel A shows an example of a thread in which the top-level post links to somewhere else on the Internet and a comment replying to the post includes a link to a fact-checking organization. Panel B shows the distribution of threads that had been fact-checked somewhere on Reddit across subreddits, broken out by veracity and whether or not a fact-checking comment is in the thread. Panel C shows the counts of true, mixed, and false threads by month over the course of the study period.

Fig. 2.

Distributions of comments, commenters, thread depth, and post lifetime by thread veracity. Complementary cumulative distribution functions (CCDFs) of conversations of news stories that did receive a fact-checking comment (A) Size, (B) Activity, (C) Maximum depth, and (D) Lifetime.

Fig. 3.

Distribution of time to and rate of commenting prior to and after first fact-checking comment. Panel A shows the distribution of the number of minutes until the first fact-checking comment on a post was made for the three veracity ratings of posts. Panel B shows the average number of comments per minute for each minute relative to the fact-check (e.g. −20 is 20 min before the fact-checking comment is made, and 20 is 20 min after the fact-checking comment is made). We note that Panel A includes only those threads that received a fact-checking comment within 6 h of the initial post for the purposes of illustration (more than 75% of posts met this criteria).

Fig. 4.

Frequency of post deletion and removal. Panel A shows the proportion of posts that were deleted by the author separated by veracity rating and the presence or the absence of a fact-checking comment. Panel B shows the proportion of posts that were removed by the someone other than the author, typically a moderator, separated by veracity rating and the presence or the absence of a fact-checking comment. Panel C shows the proportion of posts that were deleted by the author or removed by someone other than the author separated by veracity rating and the presence or the absence of a fact-checking comment.