circ2LO: Identification of CircRNA Based on the LucaOne Large Model

Abstract

Circular RNA is a type of noncoding RNA with a special covalent bond structure. As an endogenous RNA in animals and plants, it is formed through RNA splicing. The 5' and 3' ends of the exons form circular RNA at the back-splicing sites. Circular RNA plays an important regulatory role in diseases by interacting with the associated miRNAs. Accurate identification of circular RNA can enrich the data on circular RNA and provide new ideas for drug development. At present, mainstream circular RNA recognition algorithms are divided into two categories: those based on RNA sequence position information and those based on RNA sequence biometric information. Herein, we propose a method for the recognition of circular RNA, called circ2LO, which utilizes the LucaOne large model for feature embedding of the splicing sites of RNA sequences as well as their upstream and downstream sequences to prevent semantic information loss caused by the traditional one-hot encoding method. Subsequently, it employs a convolutional layer to extract features and a self-attention mechanism to extract interactive features to accurately capture the core features of the circular RNA at the splicing sites. Finally, it uses a fully connected layer to identify circular RNA. The accuracy of circ2LO on the human dataset reached 95.47%, which is higher than the values shown by existing methods. It also achieved accuracies of 97.04% and 72.04% on the Arabidopsis and mouse datasets, respectively, demonstrating good robustness. Through rigorous validation, the circ2LO model has proven its high-precision identification capability for circular RNAs, marking it as a potentially transformative analytical platform in the circRNA research field.

Keywords: circRNA; deep learning; large language model; self-attention mechanism; transformer.

Figure 1

Overall model structure of the proposed method for circRNA identification based on the LucaOne large model. $x_1,x_2\cdots x_{n-1},x_n$ denote an embedded representation of each nucleotide in an RNA sequence.

Figure 2

Implementation process, from metadata to obtaining the vector representation of RNA sequence embeddings.

Figure 3

Schematic of the attention mechanism.

Figure 4

Results of the quantitative analysis of the key modules.

Figure 5

Effect of the number of convolutional layers on the model’s performance.

Figure 6

Effect of learning rate on model accuracy.

Figure 7

Effect of the number of linear layers on the model’s performance.