Skip to main page content
U.S. flag

An official website of the United States government

Dot gov

The .gov means it’s official.
Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you’re on a federal government site.

Https

The site is secure.
The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

Access keys NCBI Homepage MyNCBI Homepage Main Content Main Navigation
Review
. 2022;12(1):129.
doi: 10.1007/s13278-022-00951-3. Epub 2022 Sep 5.

Detection and moderation of detrimental content on social media platforms: current status and future directions

Vaishali U Gongane  1 , Mousami V Munot  1 , Alwin D Anuse  2
Affiliations
Review

Detection and moderation of detrimental content on social media platforms: current status and future directions

Vaishali U Gongane et al. Soc Netw Anal Min. 2022.

Abstract

Social Media has become a vital component of every individual's life in society opening a preferred spectrum of virtual communication which provides an individual with a freedom to express their views and thoughts. While virtual communication through social media platforms is highly desirable and has become an inevitable component, the dark side of social media is observed in form of detrimental/objectionable content. The reported detrimental contents are fake news, rumors, hate speech, aggressive, and cyberbullying which raise up as a major concern in the society. Such detrimental content is affecting person's mental health and also resulted in loss which cannot be always recovered. So, detecting and moderating such content is a prime need of time. All social media platforms including Facebook, Twitter, and YouTube have made huge investments and also framed policies to detect and moderate such detrimental content. It is of paramount importance in the first place to detect such content. After successful detection, it should be moderated. With an overflowing increase in detrimental content on social media platforms, the current manual method to identify such content will never be enough. Manual and semi-automated moderation methods have reported limited success. A fully automated detection and moderation is a need of time to come up with the alarming detrimental content on social media. Artificial Intelligence (AI) has reached across all sectors and provided solutions to almost all problems, social media content detection and moderation is not an exception. So, AI-based methods like Natural Language Processing (NLP) with Machine Learning (ML) algorithms and Deep Neural Networks is rigorously deployed for detection and moderation of detrimental content on social media platforms. While detection of such content has been receiving good attention in the research community, moderation has received less attention. This research study spans into three parts wherein the first part emphasizes on the methods to detect the detrimental components using NLP. The second section describes about methods to moderate such content. The third part summarizes all observations to provide identified research gaps, unreported problems and provide research directions.

Keywords: Artificial intelligence (AI); Detection and moderation; Natural language processing (NLP); Social media (SM) platforms.

PubMed Disclaimer

Figures

Fig.1
Fig.1
Statistics of monthly active users on various social media platforms (https://www.statista.com/statistics/278414/number-of-worldwide-social-network-users/)
Fig. 2
Fig. 2
Various forms of Detriment content published on SM
Fig. 3
Fig. 3
Online abusive behavior experienced by teenagers
Fig. 4
Fig. 4
Detection and moderation of UGC on SM platforms
Fig. 5
Fig. 5
Flow chart of selection of articles for review
Fig. 6
Fig. 6
AI-based techniques for detection of detrimental content on SM platforms
Fig. 7
Fig. 7
A generic block diagram of automated SM content detection using NLP, ML and DL
Fig. 8
Fig. 8
Process of detection and classification of a SM content using ML algorithms
Fig. 9
Fig. 9
Statistics of ML algorithms for SM content Detection
Fig. 10
Fig. 10
Example images in fake news articles
Fig. 11
Fig. 11
Architecture of EANN: Event Adversarial Neural Networks for Multi-Modal Fake News Detection (Wang et al. 2018)
Fig. 12
Fig. 12
Architecture of SpotFake (Singhal et al. 2019)
Fig. 13
Fig. 13
Neural network model architecture for multimodal hate speech classification Kumar et al. (2021)
Fig. 14
Fig. 14
Manual Content moderation on SM Platforms
See this image and copyright information in PMC

References

    1. Ahmed H, Traore I, Saad S (2017) Detection of online fake news using N-Gram analysis and machine learning techniques. In: Traore I, Woungang I, Awad A (eds) Intelligent, secure, and dependable systems in distributed and cloud environments. ISDDC. Lecture Notes in Computer Science, Vol. 10618, pp 127–138. 10.1007/978-3-319-69155-8_9.
    1. Amrutha BR, Bindu KR (2019) Detecting hate speech in tweets using different deep neural network architectures. In: Proceedings of the international conference on intelligent computing and control systems (ICICCS 2019) IEEE, pp 923–926. 10.1109/ICCS45141.2019.9065763.
    1. Andersen JS, Zukunft O, Maalej W (2021) REM: efficient semi-automated real-time moderation of online forums. In: Proceedings of the joint conference of the 59th annual meeting of the association for computational linguistics and the 11th international joint conference on natural language processing: system demonstrations. pp 142–149.
    1. Ayo FE, Folorunso O, Ibharalu FT, Osinuga IA. Machine learning techniques for hate speech classification of twitter data: State-of-the-art, future challenges and research directions. Comput Sci Rev Elsevier. 2020 doi: 10.1016/j.cosrev.2020.100311. - DOI
    1. Badjatiya P, Gupta S, Gupta M, Varma V (2017) Deep learning for hate speech detection in tweets. In: 26th international conference on world wide web companion, Perth, Australia, pp 759–760. 10.1145/3041021.3054223.

LinkOut - more resources