CircSSNN: circRNA-binding site prediction via sequence self-attention neural networks with pre-normalization

Abstract

Background: Circular RNAs (circRNAs) play a significant role in some diseases by acting as transcription templates. Therefore, analyzing the interaction mechanism between circRNA and RNA-binding proteins (RBPs) has far-reaching implications for the prevention and treatment of diseases. Existing models for circRNA-RBP identification usually adopt convolution neural network (CNN), recurrent neural network (RNN), or their variants as feature extractors. Most of them have drawbacks such as poor parallelism, insufficient stability, and inability to capture long-term dependencies.

Methods: In this paper, we propose a new method completely using the self-attention mechanism to capture deep semantic features of RNA sequences. On this basis, we construct a CircSSNN model for the cirRNA-RBP identification. The proposed model constructs a feature scheme by fusing circRNA sequence representations with statistical distributions, static local contexts, and dynamic global contexts. With a stable and efficient network architecture, the distance between any two positions in a sequence is reduced to a constant, so CircSSNN can quickly capture the long-term dependencies and extract the deep semantic features.

Results: Experiments on 37 circRNA datasets show that the proposed model has overall advantages in stability, parallelism, and prediction performance. Keeping the network structure and hyperparameters unchanged, we directly apply the CircSSNN to linRNA datasets. The favorable results show that CircSSNN can be transformed simply and efficiently without task-oriented tuning.

Conclusions: In conclusion, CircSSNN can serve as an appealing circRNA-RBP identification tool with good identification performance, excellent scalability, and wide application scope without the need for task-oriented fine-tuning of parameters, which is expected to reduce the professional threshold required for hyperparameter tuning in bioinformatics analysis.

Keywords: CircRNA-binding site prediction; Circular RNAs; Pre-normalization; RNA-binding proteins; Self-attention neural networks.

Fig. 1

Encoding scheme of KNFP

Fig. 2

The network framework of the CircSSNN

Fig. 3

Experimental flowchart of the CircSSNN

Fig. 4

The structure of Seq_Transformer

Fig. 5

The average performance of competing methods on 37 circRNA datasets

Fig. 6

Comparison of the stability of HCRNet

Fig. 7

Comparison of the effect of pre-norm and post-norm on 37 circRNA datasets

Fig. 8

Comparison of the effect of different feature descriptors

Fig. 9

Boxplot comparison results of different models on 31 linear RNA datasets regarding to AUC

Fig. 10

Comparison of CircSSNN and HCRNet on 31 linear RNA datasets