Simulated-to-real benchmarking of acquisition methods in untargeted metabolomics

Abstract

Data-Dependent and Data-Independent Acquisition modes (DDA and DIA, respectively) are both widely used to acquire MS2 spectra in untargeted liquid chromatography tandem mass spectrometry (LC-MS/MS) metabolomics analyses. Despite their wide use, little work has been attempted to systematically compare their MS/MS spectral annotation performance in untargeted settings due to the lack of ground truth and the costs involved in running a large number of acquisitions. Here, we present a systematic in silico comparison of these two acquisition methods in untargeted metabolomics by extending our Virtual Metabolomics Mass Spectrometer (ViMMS) framework with a DIA module. Our results show that the performance of these methods varies with the average number of co-eluting ions as the most important factor. At low numbers, DIA outperforms DDA, but at higher numbers, DDA has an advantage as DIA can no longer deal with the large amount of overlapping ion chromatograms. Results from simulation were further validated on an actual mass spectrometer, demonstrating that using ViMMS we can draw conclusions from simulation that translate well into the real world. The versatility of the Virtual Metabolomics Mass Spectrometer (ViMMS) framework in simulating different parameters of both Data-Dependent and Data-Independent Acquisition (DDA and DIA) modes is a key advantage of this work. Researchers can easily explore and compare the performance of different acquisition methods within the ViMMS framework, without the need for expensive and time-consuming experiments with real experimental data. By identifying the strengths and limitations of each acquisition method, researchers can optimize their choice and obtain more accurate and robust results. Furthermore, the ability to simulate and validate results using the ViMMS framework can save significant time and resources, as it eliminates the need for numerous experiments. This work not only provides valuable insights into the performance of DDA and DIA, but it also opens the door for further advancements in LC-MS/MS data acquisition methods.

Keywords: data independent acquisition; data-dependent acquisition; digital twin; liquid chromatography tandem mass spectrometry; metabolomics.

FIGURE 1

The overall schematic of the ViMMS framework. (A) The Simulated Environment in ViMMS allows for new acquisition methods to be developed against a Virtual MS that takes simulated molecules as input. The new DIA methods, e.g., SWATH and AIF (solid purple box), as well as existing DDA methods (faded orange box), are implemented as controllers and initially tested here in the Simulated Environment. A controller is a specific Python implementation of an acquisition method in ViMMS. (B) Acquisition methods can be run for method validation on the Real Environment in ViMMS, connected to a Thermo Orbitrap Fusion instrument via IAPI, to acquire real experimental scans. Controllers developed in the Simulated Environment can be transferred to the Real Environment easily (shown by the dashed purple line). As different environments abstract the low-level scan generation process, the underlying Python controller codes for the new DIA methods remain unchanged when transferred from the Simulated to the Real Environment (dashed purple box).

FIGURE 2

The mean proportion of unique chemical annotations at varying numbers of chemicals and similarity thresholds across five replicates. The error bar shows the 95% confidence interval.

FIGURE 3

The distribution of cosine similarity scores when matching observed spectra to the true reference spectra for varying numbers of chemicals.

FIGURE 4

The distribution of cosine similarity scores as a boxplot (left) and a histogram (right) when matching observed spectra to the true reference spectra for 5,000 chemicals.

FIGURE 5

The distribution of pairwise cosine similarity scores of the ground truth (true chemical spectra), and fragmentation spectra from Top-N, SWATH and AIF. The plot y-axes are truncated at 25% similarity.

FIGURE 6

Proportion of matched features to the total number of detected features from the fullscan data.

FIGURE 7

Pairwise similarities of spectra in each dataset. DIA methods (AIF, SWATH) produce deconvoluted spectra that are more similar to each other.

FIGURE 8

Distribution of cosine similarity of annotated features for Top-N and DIA methods for the (A) GNPS/NIST14 library and (B) Multi-Injection library.

FIGURE 9

Venn diagram showing the overlap of annotated features between Top-N, SWATH and AIF at matching threshold ≥60% for the (A) GNPS/NIST14 library, and (B) Multi-Injection library.