Skip to main page content
U.S. flag

An official website of the United States government

Dot gov

The .gov means it’s official.
Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you’re on a federal government site.

Https

The site is secure.
The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

Access keys NCBI Homepage MyNCBI Homepage Main Content Main Navigation
Review
. 2023;30(4):2761-2775.
doi: 10.1007/s11831-023-09884-2. Epub 2023 Jan 20.

Self-supervised Learning: A Succinct Review

Veenu Rani  1 , Syed Tufael Nabi  1 , Munish Kumar  1 , Ajay Mittal  2 , Krishan Kumar  2
Affiliations
Review

Self-supervised Learning: A Succinct Review

Veenu Rani et al. Arch Comput Methods Eng. 2023.

Abstract

Machine learning has made significant advances in the field of image processing. The foundation of this success is supervised learning, which necessitates annotated labels generated by humans and hence learns from labelled data, whereas unsupervised learning learns from unlabeled data. Self-supervised learning (SSL) is a type of un-supervised learning that helps in the performance of downstream computer vision tasks such as object detection, image comprehension, image segmentation, and so on. It can develop generic artificial intelligence systems at a low cost using unstructured and unlabeled data. The authors of this review article have presented detailed literature on self-supervised learning as well as its applications in different domains. The primary goal of this review article is to demonstrate how images learn from their visual features using self-supervised approaches. The authors have also discussed various terms used in self-supervised learning as well as different types of learning, such as contrastive learning, transfer learning, and so on. This review article describes in detail the pipeline of self-supervised learning, including its two main phases: pretext and downstream tasks. The authors have shed light on various challenges encountered while working on self-supervised learning at the end of the article.

Keywords: Contrastive learning; Machine learning; Self-supervised; Supervised learning; Un-supervised learning.

PubMed Disclaimer

Conflict of interest statement

Conflict of interestThe authors declare that they have no conflict of interest in this work.

Figures

Fig. 1
Fig. 1
Self-supervised learning process
Fig. 2
Fig. 2
Supervised learning
Fig. 3
Fig. 3
Semi-supervised learning
Fig. 4
Fig. 4
Unsupervised learning
Fig. 5
Fig. 5
Reinforcement learning
Fig. 6
Fig. 6
Incremental learning
Fig. 7
Fig. 7
Block diagram of SSL
Fig. 8
Fig. 8
Various tasks performed by a pretext and b downstream
Fig. 9
Fig. 9
Several exapmples of pretext task (a) colorizing the image (b) Predicting a missing patch (c) Estimating the rotation angle (d) Jigsaw Puzzle
Fig. 10
Fig. 10
Contrastive learning in computer vision
Fig. 11
Fig. 11
Pseudo labeling process
See this image and copyright information in PMC

References

    1. Albelwi S. Survey on self-supervised learning: auxiliary pretext tasks and contrastive learning methods in imaging. Entropy. 2022;24(4):551. doi: 10.3390/e24040551. - DOI - PMC - PubMed
    1. Ohri K, Kumar M. Review on self-supervised image recognition using deep neural networks. Knowl-Based Syst. 2021;224:7090. doi: 10.1016/j.knosys.2021.107090. - DOI
    1. Orhan, AE, Gupta VV, Lake BM (2007) Self-supervised learning through the eues of a child 2020, arXiv e-prints, arXiv-2007
    1. Tao L, Wang X, Yamasaki T. An improved inter-intra contrastive learning framework on self-supervised video representation. IEEE Trans Circ Syst Video Technol. 2022 doi: 10.1109/tcsvt.2022.3141051. - DOI
    1. Jaiswal A, Babu AR, Zadeh MZ, Banerjee D, Makedon F. A survey on contrastive self-supervised learning. Technologies. 2020;9(1):2. doi: 10.3390/technologies9010002. - DOI

LinkOut - more resources