QUAREP-LiMi: A community-driven initiative to establish guidelines for quality assessment and reproducibility for instruments and images in light microscopy

Abstract

A modern day light microscope has evolved from a tool devoted to making primarily empirical observations to what is now a sophisticated , quantitative device that is an integral part of both physical and life science research. Nowadays, microscopes are found in nearly every experimental laboratory. However, despite their prevalent use in capturing and quantifying scientific phenomena, neither a thorough understanding of the principles underlying quantitative imaging techniques nor appropriate knowledge of how to calibrate, operate and maintain microscopes can be taken for granted. This is clearly demonstrated by the well-documented and widespread difficulties that are routinely encountered in evaluating acquired data and reproducing scientific experiments. Indeed, studies have shown that more than 70% of researchers have tried and failed to repeat another scientist's experiments, while more than half have even failed to reproduce their own experiments. One factor behind the reproducibility crisis of experiments published in scientific journals is the frequent underreporting of imaging methods caused by a lack of awareness and/or a lack of knowledge of the applied technique. Whereas quality control procedures for some methods used in biomedical research, such as genomics (e.g. DNA sequencing, RNA-seq) or cytometry, have been introduced (e.g. ENCODE), this issue has not been tackled for optical microscopy instrumentation and images. Although many calibration standards and protocols have been published, there is a lack of awareness and agreement on common standards and guidelines for quality assessment and reproducibility. In April 2020, the QUality Assessment and REProducibility for instruments and images in Light Microscopy (QUAREP-LiMi) initiative was formed. This initiative comprises imaging scientists from academia and industry who share a common interest in achieving a better understanding of the performance and limitations of microscopes and improved quality control (QC) in light microscopy. The ultimate goal of the QUAREP-LiMi initiative is to establish a set of common QC standards, guidelines, metadata models and tools, including detailed protocols, with the ultimate aim of improving reproducible advances in scientific research. This White Paper (1) summarizes the major obstacles identified in the field that motivated the launch of the QUAREP-LiMi initiative; (2) identifies the urgent need to address these obstacles in a grassroots manner, through a community of stakeholders including, researchers, imaging scientists, bioimage analysts, bioimage informatics developers, corporate partners, funding agencies, standards organizations, scientific publishers and observers of such; (3) outlines the current actions of the QUAREP-LiMi initiative and (4) proposes future steps that can be taken to improve the dissemination and acceptance of the proposed guidelines to manage QC. To summarize, the principal goal of the QUAREP-LiMi initiative is to improve the overall quality and reproducibility of light microscope image data by introducing broadly accepted standard practices and accurately captured image data metrics.

Keywords: confocal; light microscopy; metadata; quality assessment; quality control; reproducibility; widefield.

FIGURE 1

Acquiring imaging data that is both quantifiable and reproducible involves a myriad of factors, few of which are acknowledged or accurately recorded Note: Intimate knowledge of the composition and performance of a system is essential for reproducibility. However, performance measurements may be tricky, and require specific protocols, tools, samples, training and data analysis methods. In order to help microscope users to assess and judge the performance of their systems properly, the community must agree on and publish guidelines and benchmarks.

FIGURE 2

Frequency and type of tests performed in core facilities Note: Prior to a session on QC organized during the Microscopy Facility Day at the 2019 ELMI meeting, light microscopy core facility representatives around the globe were asked to complete a survey about the type and the frequency of tests performed in their facility. The link to the survey was opened in June 2019, 2 weeks before the meeting, and sent to all registered participants. It was also advertized multiple times on various international microscopy forums. Reminders were also sent to participants after the meeting and responses were subsequently collected until February 2020. The histograms in panels (A) and (B) summarize the responses from almost 200 facilities in a simplified manner. Panel (A) displays how often different quality checks are performed; the x-axis represents in percentage the respective frequency categories, namely, regularly, on demand and never. Panel (B) highlights which tools are used for performance evaluation of light microscopes; the x-axis represents the percentage of respondents using the indicated QC tool.

FIGURE 3

Summary of current QUAREP-LiMi participants according to their origin and affiliation with the QUAREP-LiMi logo (updated: 13 May 2021)