Reduced speech coherence in psychosis-related social media forum posts

Abstract

The extraction of linguistic markers from social media posts, which are indicative of the onset and course of mental disorders, offers great potential for mental healthcare. In the present study, we extracted over one million posts from the popular social media platform Reddit to analyze speech coherence, which reflects formal thought disorder and is a characteristic feature of schizophrenia and associated psychotic disorders. Natural language processing (NLP) models were used to perform an automated quantification of speech coherence. We could demonstrate that users who are active on forums geared towards disorders with a higher degree of psychotic symptoms tend to show a lower level of coherence. The lowest coherence scores were found in users of forums on dissociative identity disorder, schizophrenia, and bipolar disorder. In contrast, a relatively high level of coherence was detected in users of forums related to obsessive-compulsive disorder, anxiety, and depression. Users of forums on posttraumatic stress disorder, autism, and attention-deficit hyperactivity disorder exhibited medium-level coherence. Our findings provide promising first evidence for the possible utility of NLP-based coherence analyses for the early detection and prevention of psychosis on the basis of posts gathered from publicly available social media data. This opens new avenues for large-scale prevention programs aimed at high-risk populations.

Fig. 1. Coherence calculation process.

Note. Paragraphs were first divided into sentences (tokenization). Afterward, sentences were embedded into 512-dimensional semantic space using Google’s Universal Sentence Encoder (GUSE) (Cer et al., 2018). The inner product of the embeddings of subsequent sentences represents their semantic similarity. The average of all sentence pairs’ similarity yielded the coherence scores. d₁s₁ denotes the first embedding dimension of the embedding vector of the first sentence. The narrative is quite coherent until the last sentence pair which is marked by a lower semantic similarity and thus represents a break in coherence.

Fig. 2. Estimated marginal means for the coherence scores of all ten subreddits.

Note. Error bars represent the standard error of the estimated marginal means. The control group is composed of a sample of posts derived from the subreddits “r/self”, “r/relationship_advice”, “r/dating_advice”, “r/pettyrevenge” and “r/socialskills”.

Fig. 3. Estimated marginal means for the coherence scores of posts made in neutral subreddits.

Note. Error bars represent the standard error of the estimated marginal means. The control group is composed of posts made by users of the subreddits “r/self”, “r/relationship_advice”, “r/dating_advice”, “r/pettyrevenge” and “r/socialskills”. The sample sizes after filtration were N_control = 121,897, N_Anxiety = 35,108, N_depression = 26,499, N_OCD = 30,208, N_ADHD = 22,967, N_autism = 30,818, N_ptsd = 30,533, N_{schizophrenia} = 23,026, N_bipolar = 20,152, N_DID = 12,405.