Multilaboratory Untargeted Mass Spectrometry Metabolomics Collaboration to Identify Bottlenecks and Comprehensively Annotate A Single Dataset

Abstract

Annotation is the process of assigning features in mass spectrometry metabolomics data sets to putative chemical structures or "analytes." The purpose of this study was to identify challenges in the annotation of untargeted mass spectrometry metabolomics datasets and suggest strategies to overcome them. Toward this goal, we analyzed an extract of the plant ashwagandha (Withania somnifera) using liquid chromatography-mass spectrometry on two different platforms (an Orbitrap and Q-ToF) with various acquisition modes. The resulting 12 datasets were shared with ten teams that had established expertise in metabolomics data interpretation. Each team annotated at least one positive ion dataset using their own approaches. Eight teams selected the positive ion mode data-dependent acquisition (DDA) data collected on the Orbitrap platform, so the results reported for that dataset were chosen for an in-depth comparison. We compiled and cross-checked the annotations of this dataset from each laboratory to arrive at a "consensus annotation," which included 142 putative analytes, of which 13 were confirmed by comparison with standards. Each team only reported a subset (24 to 57%) of the analytes in the consensus list. Correct assignment of ion species (clusters and fragments) in MS spectra was a major bottleneck. In many cases, in-source redundant features were mistakenly considered to be independent analytes, causing annotation errors and resulting in overestimation of sample complexity. Our results suggest that better tools/approaches are needed to effectively assign feature identity, group related mass features, and query published spectral and taxonomic data when assigning putative analyte structures.

1

Annotation Agreement Scores assess the feature-level agreement between the 8 teams for the Orbitrap data-dependent analysis (DDA) positive ionization dataset. Assignment Match (j) is defined as the number of teams that agreed on the assignment for a given Element. Four Elements of the annotations were considered: the “feature reported” element (defined by m/z and retention time), the “ion species description” element (ion species, [M + H]⁺, [M + Na]⁺, etc.), the “chemical class” element (defined by the NP classifier), and the “identity” element (two-dimensional identity). The Annotation Agreement Score is calculated from eq and described in more detail in SI-01 Figure S18. As an example, for Assignment Match 2 and the “Chemical class” element, an Annotation Agreement Score of 58 indicates that 58 features were assigned to the same chemical class by two teams.

2

Histogram of the ion species description proposed by the 8 teams for the features reported in the Merged Annotation Table compared to the assignments in the final Consensus Annotation Table. The results indicate that teams were biased toward [M + H]⁺ ions in their original assignments (black bars) and that reassignment (blue bars) caused a shift toward other ion species (see SI-01 Figures S25–S26).

3

Consensus Annotation Score analysis (SI-01 Figure S21) assesses the overlap between the lists provided by the 8 teams and the Consensus Annotation Table. A total of 142 analytes (unique chemical entities) were annotated in the Consensus Annotation Table after grouping all of the features. “Analyte” refers to the number of analytes reported by a given team that overlaps with those included in the Consensus Annotation Table, while “Ion species,” “Chemical class,” and “Identity” refer to the number of analytes with these elements that match the elements in the Consensus Annotation Table. For example, in Panel A, Team 1 reported 47 (33%) of the 142 analytes included in the Consensus Annotation Table, and 32, 41, and 23 of these analytes, respectively, were reported to have the same ion species, chemical class, and identity assigned in the Consensus Annotation Table. Panel A shows the analyses for all analytes, and panel B for the 13 confirmed by comparison with standards.

4

LC-MS data and annotation of withanone (confidence level 1A). (A) Structure of withanone, (B) MS spectrum of the pure standard, (C) Ion species description illustrated with a bar plot that represents the peak area of the features associated with one analyte as a function of mass-to-charge ratio (m/z), (D) MS spectrum of withanone in the Withania somnifera extract, and (E) annotation of its features and their detection by the 8 teams. [M+X]⁺ means the ion species, and ID means its identity (“x” false annotation, “√” correct annotation, “c” correct chemical class but false annotation; blue fill color means correspondence with the consensus annotation, red fill color means no correspondence). The features in red font were not reported by any teams but were detected in our manual interpretation and by comparison with standards, and the ones in blue font did not have any MS/MS spectra. See SI-01 Figure S35 for the fragmentation spectra.