Detection and moderation of detrimental content on social media platforms: current status and future directions

Abstract

Social Media has become a vital component of every individual's life in society opening a preferred spectrum of virtual communication which provides an individual with a freedom to express their views and thoughts. While virtual communication through social media platforms is highly desirable and has become an inevitable component, the dark side of social media is observed in form of detrimental/objectionable content. The reported detrimental contents are fake news, rumors, hate speech, aggressive, and cyberbullying which raise up as a major concern in the society. Such detrimental content is affecting person's mental health and also resulted in loss which cannot be always recovered. So, detecting and moderating such content is a prime need of time. All social media platforms including Facebook, Twitter, and YouTube have made huge investments and also framed policies to detect and moderate such detrimental content. It is of paramount importance in the first place to detect such content. After successful detection, it should be moderated. With an overflowing increase in detrimental content on social media platforms, the current manual method to identify such content will never be enough. Manual and semi-automated moderation methods have reported limited success. A fully automated detection and moderation is a need of time to come up with the alarming detrimental content on social media. Artificial Intelligence (AI) has reached across all sectors and provided solutions to almost all problems, social media content detection and moderation is not an exception. So, AI-based methods like Natural Language Processing (NLP) with Machine Learning (ML) algorithms and Deep Neural Networks is rigorously deployed for detection and moderation of detrimental content on social media platforms. While detection of such content has been receiving good attention in the research community, moderation has received less attention. This research study spans into three parts wherein the first part emphasizes on the methods to detect the detrimental components using NLP. The second section describes about methods to moderate such content. The third part summarizes all observations to provide identified research gaps, unreported problems and provide research directions.

Keywords: Artificial intelligence (AI); Detection and moderation; Natural language processing (NLP); Social media (SM) platforms.

Fig.1

Statistics of monthly active users on various social media platforms (https://www.statista.com/statistics/278414/number-of-worldwide-social-network-users/)

Fig. 2

Various forms of Detriment content published on SM

Fig. 3

Online abusive behavior experienced by teenagers

Fig. 4

Detection and moderation of UGC on SM platforms

Fig. 5

Flow chart of selection of articles for review

Fig. 6

AI-based techniques for detection of detrimental content on SM platforms

Fig. 7

A generic block diagram of automated SM content detection using NLP, ML and DL

Fig. 8

Process of detection and classification of a SM content using ML algorithms

Fig. 9

Statistics of ML algorithms for SM content Detection

Fig. 10

Example images in fake news articles

Fig. 11

Architecture of EANN: Event Adversarial Neural Networks for Multi-Modal Fake News Detection (Wang et al. 2018)

Fig. 12

Architecture of SpotFake (Singhal et al. 2019)

Fig. 13

Neural network model architecture for multimodal hate speech classification Kumar et al. (2021)

Fig. 14

Manual Content moderation on SM Platforms