Transformer-based tool recommendation system in Galaxy

Abstract

Background: Galaxy is a web-based open-source platform for scientific analyses. Researchers use thousands of high-quality tools and workflows for their respective analyses in Galaxy. Tool recommender system predicts a collection of tools that can be used to extend an analysis. In this work, a tool recommender system is developed by training a transformer on workflows available on Galaxy Europe and its performance is compared to other neural networks such as recurrent, convolutional and dense neural networks.

Results: The transformer neural network achieves two times faster convergence, has significantly lower model usage (model reconstruction and prediction) time and shows a better generalisation that goes beyond training workflows than the older tool recommender system created using RNN in Galaxy. In addition, the transformer also outperforms CNN and DNN on several key indicators. It achieves a faster convergence time, lower model usage time, and higher quality tool recommendations than CNN. Compared to DNN, it converges faster to a higher precision@k metric (approximately 0.98 by transformer compared to approximately 0.9 by DNN) and shows higher quality tool recommendations.

Conclusion: Our work shows a novel usage of transformers to recommend tools for extending scientific workflows. A more robust tool recommendation model, created using a transformer, having significantly lower usage time than RNN and CNN, higher precision@k than DNN, and higher quality tool recommendations than all three neural networks, will benefit researchers in creating scientifically significant workflows and exploratory data analysis in Galaxy. Additionally, the ability to train faster than all three neural networks imparts more scalability for training on larger datasets consisting of millions of tool sequences. Open-source scripts to create the recommendation model are available under MIT licence at https://github.com/anuprulez/galaxy_tool_recommendation_transformers.

Keywords: Artificial intelligence; Galaxy; Recommendation system; Tools; Transformer; Workflows.

Fig. 1

Neural network architecture of transformer used for recommending tools in Galaxy. Figure 1a–b represents how a sequence of tools is transformed into a sequence of integers. Figure 1c–g represents several different neural network layers through which sequences of tools are passed to learn their respective representations and mapping with their respective labels. Figure 1h represents the output in the form of a real-valued vector

Fig. 2

A comparison of precision@k metric, used for evaluating a recommender system, is shown over training iterations for the transformer (green), RNN (red), CNN (blue) and DNN (black) architectures. The precision@k values are averaged over 5 experiment runs for each architecture and are shown as line plots. The shaded regions show the standard deviation across 5 experiment runs

Fig. 3

A comparison of the model usage (model reconstruction + prediction) time of transformer (green), RNN (red), CNN (blue) and DNN (black) is shown. The time is averaged over 5 experiment runs, and the shaded region shows the standard deviation across 5 experiment runs

Fig. 4

The self-attention weights for a tool sequence, read from left to right on the horizontal axis or top to bottom on the vertical axis, from the tool suite ChemicalToolBox (CTB) [42] in Galaxy Europe are shown. It is seen that tools from the CTB suite (containing “ctb” prefix) attend to each other as they have higher correlation weights, but they don’t attend to the “show beginning1” tool, which is only a text formatting tool and not from the CTB suite

Fig. 5

The self-attention weights for a tool sequence, read from left to right on the horizontal axis or top to bottom on the vertical axis and used for performing differential expression analysis, are shown. It is seen that tools such as trimmomatic [45], hisat2 [46], featurecounts [47] and deseq2 [48] are more correlated to each other compared to the text formatting tools such as “join1” or “filter1”