Automatic detection of depression symptoms in twitter using multimodal analysis

Abstract

Depression is the most prevalent mental disorder that can lead to suicide. Due to the tendency of people to share their thoughts on social platforms, social data contain valuable information that can be used to identify user's psychological states. In this paper, we provide an automated approach to collect and evaluate tweets based on self-reported statements and present a novel multimodal framework to predict depression symptoms from user profiles. We used n-gram language models, LIWC dictionaries, automatic image tagging, and bag-of-visual-words. We consider the correlation-based feature selection and nine different classifiers with standard evaluation metrics to assess the effectiveness of the method. Based on the analysis, the tweets and bio-text alone showed 91% and 83% accuracy in predicting depressive symptoms, respectively, which seems to be an acceptable result. We also believe performance improvements can be achieved by limiting the user domain or presence of clinical information.

Keywords: Depression detection; Mental health; Multimodal framework; Social media; Text mining.

Fig. 1

The growing number of US youths with a major depressive episode from 2004 to 2019, by gender [6]

Fig. 2

The high-level architecture of the proposed framework consists of three main modules: 1) data collection and dataset building, 2) cross-analysis, and 3) classification

Fig. 3

The confirmed range of the polarity score, after initial filtering

Fig. 4

The structure of the term-document in the case study (T = Term, D = Document, W = Weight)

Fig. 5

The proportion of distinct LIWC categories

Fig. 5

The proportion of distinct LIWC categories

Fig. 6

Unigrams and bigrams word cloud

Fig. 7

The Pearson correlation heatmap among the LIWC dictionaries for both $D_{\mathit{tweets}}^{+}$ and $D_{\mathit{tweets}}^{-}$

Fig. 8

Comparison of the highest accuracy achieved on feature types by nine different classifiers

Fig. 9

Comparison of the highest accuracy achieved by tweets analyzing methods

Fig. 10

Comparison of the highest accuracy achieved by bios analyzing methods

Fig. 11

ROC curves of the target classifiers

Fig. 12

Comparison of the highest accuracy achieved by tweets analyzing methods via SVD approach

Fig. 13

ROC curves of the target classifiers for SVD analysis