Scoring alignments by embedding vector similarity

Abstract

Sequence similarity is of paramount importance in biology, as similar sequences tend to have similar function and share common ancestry. Scoring matrices, such as PAM or BLOSUM, play a crucial role in all bioinformatics algorithms for identifying similarities, but have the drawback that they are fixed, independent of context. We propose a new scoring method for amino acid similarity that remedies this weakness, being contextually dependent. It relies on recent advances in deep learning architectures that employ self-supervised learning in order to leverage the power of enormous amounts of unlabelled data to generate contextual embeddings, which are vector representations for words. These ideas have been applied to protein sequences, producing embedding vectors for protein residues. We propose the E-score between two residues as the cosine similarity between their embedding vector representations. Thorough testing on a wide variety of reference multiple sequence alignments indicate that the alignments produced using the new $E$-score method, especially ProtT5-score, are significantly better than those obtained using BLOSUM matrices. The new method proposes to change the way alignments are computed, with far-reaching implications in all areas of textual data that use sequence similarity. The program to compute alignments based on various $E$-scores is available as a web server at e-score.csd.uwo.ca. The source code is freely available for download from github.com/lucian-ilie/E-score.

Keywords: alignment distance; amino acid scoring matrices; sequence alignment; sequence similarity; word embedding.

Figure 1

Cosine similarity of vectors and is the cosine of the angle, , between them: (A) similar vectors: , ; (B) orthogonal (independent) vectors: , ; (C) opposite vectors: , .

Figure 2

Heatmaps of (A) BLOSUM45 matrix (scaled to ) and three aligned matrices of average -scores for the NBD_sugar-kinase_HSP70_actin MSA: (B) ProtT5-score, (C) ESM2-score and (D) ProtAlbert-score.

Figure 3

Pearson correlations between 5 BLOSUM matrices and 12 -score matrices. For each embedding method we considered, the aligned and unaligned matrices for the NBD_sugar-kinase_HSP70_actin MSA have been included.

Figure 4

(A) The ratio between the performance of BLOSUM45 and ProtT5-score in terms of distances to the reference from their alignments. The results are sorted by increasing MSA length. Higher than 1 (above the line) indicates ProtT5-score is better. (B) The percentage of cases when ProtT5-score and BLOSUM45, respectively, are producing better results, sorted increasingly by MSA length. The results for the same distance are plotted in the same colour, with solid line for ProtT5-score and dashed line for BLOSUM45.