A framework to assess the quality and impact of bioinformatics training across ELIXIR

Abstract

ELIXIR is a pan-European intergovernmental organisation for life science that aims to coordinate bioinformatics resources in a single infrastructure across Europe; bioinformatics training is central to its strategy, which aims to develop a training community that spans all ELIXIR member states. In an evidence-based approach for strengthening bioinformatics training programmes across Europe, the ELIXIR Training Platform, led by the ELIXIR EXCELERATE Quality and Impact Assessment Subtask in collaboration with the ELIXIR Training Coordinators Group, has implemented an assessment strategy to measure quality and impact of its entire training portfolio. Here, we present ELIXIR's framework for assessing training quality and impact, which includes the following: specifying assessment aims, determining what data to collect in order to address these aims, and our strategy for centralised data collection to allow for ELIXIR-wide analyses. In addition, we present an overview of the ELIXIR training data collected over the past 4 years. We highlight the importance of a coordinated and consistent data collection approach and the relevance of defining specific metrics and answer scales for consortium-wide analyses as well as for comparison of data across iterations of the same course.

Fig 1. ELIXIR’s training metrics database.

The database may be publicly accessed at https://training-metrics-dev.elixir-europe.org/. (A) A summary of ELIXIR-wide training may be viewed on the ‘Dashboard’. (B) A list of all ELIXIR training events may be viewed on the ‘All ELIXIR’ events page. (C) Users may generate interactive reports on the ‘Reports’ page. (Screenshots were taken on 6 August 2019).

Fig 2. Why did you attend the training?.

Past participants reflected on their reasons for attending the training. Graph generated in the R environment using the tidyverse package [12] and ggplot2 theme adapted from https://orchid00.github.io/ElixirBE/reports/2018March_nt.

Fig 3. How did the training help with your work?.

Each respondent was able to select multiple answers to this question. The ‘upset diagram’ illustrates how participants felt that the training has helped them with their work (‘Response Per Option (set size)’, lower LHS) as well as the different combinations of answers selected by the same individual, represented by the size of the intersection (‘Intersection Size’, RHS, ‘orange’). Packages used to create this graph (and graph in Fig 4) in the R environment were tidyverse [12] and UpsetR [13]. LHS, left-hand side; RHS, right-hand side.

Fig 4. Did the training lead to or facilitate any of the following outcomes?.

Each respondent was able to select multiple answers to this question. The ‘upset diagram’ illustrates what outcomes the training has led to or facilitated (‘Response Per Option (set size)’, lower LHS) as well as the different combinations of answers selected by the same individual, represented by the size of the intersection (‘Intersection Size’, RHS, ‘orange’). LHS, left-hand side; RHS, right-hand side.