External Quality Assessment of SARS-CoV-2 Sequencing: an ESGMD-SSM Pilot Trial across 15 European Laboratories

Fanny Wegner^#^{1

2

3}, Tim Roloff^#^{1

2

3}, Michael Huber⁴, Samuel Cordey⁵, Alban Ramette^{6

7}, Yannick Gerth⁸, Claire Bertelli^{9

10}, Madlen Stange^{1

2

3}, Helena M B Seth-Smith^{1

2

3

7}, Alfredo Mari^{1

2

3}, Karoline Leuzinger^{11

12}, Lorenzo Cerutti¹³, Keith Harshman¹³, Ioannis Xenarios¹³, Philippe Le Mercier¹⁴, Pascal Bittel⁶, Stefan Neuenschwander⁶, Onya Opota^{9

10

7}, Jonas Fuchs¹⁵, Marcus Panning¹⁵, Charlotte Michel¹⁶, Marie Hallin¹⁶, Thomas Demuyser¹⁷, Ricardo De Mendonca¹⁸, Paul Savelkoul^{19

7}, Jozef Dingemans¹⁹, Brian van der Veer¹⁹, Stefan A Boers²⁰, Eric C J Claas^{20

7}, Jordy P M Coolen²¹, Willem J G Melchers^{21

7}, Marianne Gunell^{22

23}, Teemu Kallonen^{22

23}, Tytti Vuorinen^{22

23}, Antti J Hakanen^{22

23}, Eva Bernhoff²⁴, Marit Andrea Klokkhammer Hetland²⁴, Hadar Golan Berman^{25

26}, Sheera Adar²⁶, Jacob Moran-Gilad^{27

7}, Dana G Wolf^{25

28}, Stephen L Leib^{6

7}, Oliver Nolte⁸, Laurent Kaiser⁵, Stefan Schmutz⁴, Verena Kufner⁴, Maryam Zaheri⁴, Alexandra Trkola⁴, Hege Vangstein Aamot^{29

30

7}, Hans H Hirsch^{12

31

32}, Gilbert Greub^#^{9

10

7}, Adrian Egli^#^{1

2

7}

Affiliations

¹ Applied Microbiology Research, Department of Biomedicine, University of Basel, Basel, Switzerland.
² Clinical Bacteriology and Mycology, University Hospital Basel, Basel, Switzerland.
³ Swiss Institute of Bioinformatics, Basel, Switzerland.
⁴ Institute of Medical Virology, University of Zurichgrid.7400.3, Zurich, Switzerland.
⁵ Laboratory of Virology, University Hospital Geneva, Geneva, Switzerland.
⁶ Institute for Infectious Diseases, University of Berngrid.5734.5, Bern, Switzerland.
⁷ ESCMID Study Group for Genomic and Molecular Diagnostics (ESGMD), Basel, Switzerland.
⁸ Center for Laboratory Medicine, Saint Gall, Switzerland.
⁹ Institute of Microbiology, Lausanne University Hospital, Lausanne, Switzerland.
¹⁰ Institute of Microbiology, University of Lausanne, Lausanne, Switzerland.
¹¹ Clinical Virology, University Hospital Basel, Basel, Switzerland.
¹² Transplantation and Clinical Virology, Department of Biomedicine, University of Basel, Basel, Switzerland.
¹³ Health 2030 Genome Center, Geneva, Switzerland.
¹⁴ SwissProt Group, Swiss Institute of Bioinformatics, Geneva, Switzerland.
¹⁵ Institute of Virology, Medical Center-University of Freiburg, Faculty of Medicine, University of Freiburg, Freiburg, Germany.
¹⁶ Department of Microbiology, Laboratoire Hospitalier Universitaire de Bruxelles, Brussels, Belgium.
¹⁷ Department of Microbiology and Infection Control, Universitair Ziekenhuis Brussel, Vrije Universiteit Brussel, Brussels, Belgium.
¹⁸ Université Libre de Bruxelles (ULB), Brussels, Belgium.
¹⁹ Department of Medical Microbiology, Maastricht University, Maastricht, Netherlands.
²⁰ Department of Medical Microbiology, Leiden University Medical Center, Leiden, Netherlands.
²¹ Department of Medical Microbiology, Radboud University Medical Center, Nijmegen, Netherlands.
²² Department of Clinical Microbiology, Turku University Hospital, Turku, Finland.
²³ Department of Clinical Microbiology, University of Turku, Turku, Finland.
²⁴ Department of Medical Microbiology, Stavanger University Hospital, Stavanger, Norway.
²⁵ Clinical Virology Unit, Department of Clinical Microbiology and Infectious Diseases, Hadassah University Hospital, Jerusalem, Israel.
²⁶ Department of Microbiology and Molecular Genetics, Institute for Medical Research Israel Canada, The Hebrew University Faculty of Medicine, Jerusalem, Israel.
²⁷ School of Public Health, Faculty of Health Sciences, Ben-Gurion University of the Negev, Beer Sheva, Israel.
²⁸ Lautenberg Center for General and Tumor Immunology, The Hebrew University Faculty of Medicine, Jerusalem, Israel.
²⁹ Department of Microbiology and Infection Control, Akershus University Hospitalgrid.411279.8, Lørenskog, Norway.
³⁰ Department of Clinical Molecular Biology (EPIGEN), Akershus University Hospitalgrid.411279.8 and University of Oslo, Lørenskog, Norway.
³¹ Infectious Diseases and Hospital Epidemiology, University Hospital Basel, Basel, Switzerland.
³² Infectious Diseases and Hospital Epidemiology, University of Basel, Basel, Switzerland.

^# Contributed equally.

PMID: 34757834
PMCID: PMC8769736
DOI: 10.1128/JCM.01698-21

External Quality Assessment of SARS-CoV-2 Sequencing: an ESGMD-SSM Pilot Trial across 15 European Laboratories

Fanny Wegner et al. J Clin Microbiol. 2022.

. 2022 Jan 19;60(1):e0169821.

doi: 10.1128/JCM.01698-21. Epub 2021 Nov 10.

Authors

Affiliations

¹ Applied Microbiology Research, Department of Biomedicine, University of Basel, Basel, Switzerland.
² Clinical Bacteriology and Mycology, University Hospital Basel, Basel, Switzerland.
³ Swiss Institute of Bioinformatics, Basel, Switzerland.
⁴ Institute of Medical Virology, University of Zurichgrid.7400.3, Zurich, Switzerland.
⁵ Laboratory of Virology, University Hospital Geneva, Geneva, Switzerland.
⁶ Institute for Infectious Diseases, University of Berngrid.5734.5, Bern, Switzerland.
⁷ ESCMID Study Group for Genomic and Molecular Diagnostics (ESGMD), Basel, Switzerland.
⁸ Center for Laboratory Medicine, Saint Gall, Switzerland.
⁹ Institute of Microbiology, Lausanne University Hospital, Lausanne, Switzerland.
¹⁰ Institute of Microbiology, University of Lausanne, Lausanne, Switzerland.
¹¹ Clinical Virology, University Hospital Basel, Basel, Switzerland.
¹² Transplantation and Clinical Virology, Department of Biomedicine, University of Basel, Basel, Switzerland.
¹³ Health 2030 Genome Center, Geneva, Switzerland.
¹⁴ SwissProt Group, Swiss Institute of Bioinformatics, Geneva, Switzerland.
¹⁵ Institute of Virology, Medical Center-University of Freiburg, Faculty of Medicine, University of Freiburg, Freiburg, Germany.
¹⁶ Department of Microbiology, Laboratoire Hospitalier Universitaire de Bruxelles, Brussels, Belgium.
¹⁷ Department of Microbiology and Infection Control, Universitair Ziekenhuis Brussel, Vrije Universiteit Brussel, Brussels, Belgium.
¹⁸ Université Libre de Bruxelles (ULB), Brussels, Belgium.
¹⁹ Department of Medical Microbiology, Maastricht University, Maastricht, Netherlands.
²⁰ Department of Medical Microbiology, Leiden University Medical Center, Leiden, Netherlands.
²¹ Department of Medical Microbiology, Radboud University Medical Center, Nijmegen, Netherlands.
²² Department of Clinical Microbiology, Turku University Hospital, Turku, Finland.
²³ Department of Clinical Microbiology, University of Turku, Turku, Finland.
²⁴ Department of Medical Microbiology, Stavanger University Hospital, Stavanger, Norway.
²⁵ Clinical Virology Unit, Department of Clinical Microbiology and Infectious Diseases, Hadassah University Hospital, Jerusalem, Israel.
²⁶ Department of Microbiology and Molecular Genetics, Institute for Medical Research Israel Canada, The Hebrew University Faculty of Medicine, Jerusalem, Israel.
²⁷ School of Public Health, Faculty of Health Sciences, Ben-Gurion University of the Negev, Beer Sheva, Israel.
²⁸ Lautenberg Center for General and Tumor Immunology, The Hebrew University Faculty of Medicine, Jerusalem, Israel.
²⁹ Department of Microbiology and Infection Control, Akershus University Hospitalgrid.411279.8, Lørenskog, Norway.
³⁰ Department of Clinical Molecular Biology (EPIGEN), Akershus University Hospitalgrid.411279.8 and University of Oslo, Lørenskog, Norway.
³¹ Infectious Diseases and Hospital Epidemiology, University Hospital Basel, Basel, Switzerland.
³² Infectious Diseases and Hospital Epidemiology, University of Basel, Basel, Switzerland.

^# Contributed equally.

PMID: 34757834
PMCID: PMC8769736
DOI: 10.1128/JCM.01698-21

Abstract

This first pilot trial on external quality assessment (EQA) of severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2) whole-genome sequencing, initiated by the European Society of Clinical Microbiology and Infectious Diseases (ESCMID) Study Group for Genomic and Molecular Diagnostics (ESGMD) and the Swiss Society for Microbiology (SSM), aims to build a framework between laboratories in order to improve pathogen surveillance sequencing. Ten samples with various viral loads were sent out to 15 clinical laboratories that had free choice of sequencing methods and bioinformatic analyses. The key aspects on which the individual centers were compared were the identification of (i) single nucleotide polymorphisms (SNPs) and indels, (ii) Pango lineages, and (iii) clusters between samples. The participating laboratories used a wide array of methods and analysis pipelines. Most were able to generate whole genomes for all samples. Genomes were sequenced to various depths (up to a 100-fold difference across centers). There was a very good consensus regarding the majority of reporting criteria, but there were a few discrepancies in lineage and cluster assignments. Additionally, there were inconsistencies in variant calling. The main reasons for discrepancies were missing data, bioinformatic choices, and interpretation of data. The pilot EQA was overall a success. It was able to show the high quality of participating laboratories and provide valuable feedback in cases where problems occurred, thereby improving the sequencing setup of laboratories. A larger follow-up EQA should, however, improve on defining the variables and format of the report. Additionally, contamination and/or minority variants should be a further aspect of assessment.

Keywords: NGS; external quality assessment; ring trial; whole-genome sequencing.

PubMed Disclaimer

Conflict of interest statement

The authors declare no conflict of interest.

We have no conflict of interest to declare regarding SARS-CoV-2 diagnostics.

Figures

**FIG 1**
(A) Mean read depth per sample (x axis) and center (y axis). Colors have been scaled for high resolution for values of between 0 and 10,000; values larger than this are displayed in the same color. (B) Percentage of N’s in the genome per sample (x axis) and center (y axis). (C) Score for variant detection per sample (x axis) and center (y axis) as well as mean score for each center across all samples and mean score for each sample across centers (ø). The numerical values underlying each plot can be found in Tables S2 to S4 in the supplemental material.

See this image and copyright information in PMC

References

1. European Centre for Disease Prevention and Control. 2021. Sequencing of SARS-CoV-2: first update. European Centre for Disease Prevention and Control, Solna, Sweden. https://www.ecdc.europa.eu/en/publications-data/sequencing-sars-cov-2.
1. Karmarkar EN, Blanco I, Amornkul PN, DuBois A, Deng X, Moonan PK, Rubenstein BL, Miller DA, Kennedy I, Yu J, Dauterman JP, Ongpin M, Hathaway W, Hoo L, Trammell S, Dosunmu EF, Yu G, Khwaja Z, Lu W, Talai NZ, Jain S, Louie JK, Philip SS, Federman S, Masinde G, Wadford DA, Bobba N, Stoltey J, Smith A, Epson E, Chiu CY, Bennett AS, Vasquez AM, Williams T. 2021. Timely intervention and control of a novel coronavirus (COVID-19) outbreak at a large skilled nursing facility—San Francisco, California, 2020. Infect Control Hosp Epidemiol 42:1173–1180. 10.1017/ice.2020.1375. - DOI - PMC - PubMed
1. Pillay S, Giandhari J, Tegally H, Wilkinson E, Chimukangara B, Lessells R, Moosa Y, Mattison S, Gazy I, Fish M, Singh L, Khanyile KS, San JE, Fonseca V, Giovanetti M, Alcantara LC, Jr, de Oliveira T. 2020. Whole genome sequencing of SARS-CoV-2: adapting Illumina protocols for quick and accurate outbreak investigation during a pandemic. Genes (Basel) 11:949. 10.3390/genes11080949. - DOI - PMC - PubMed
1. Stange M, Mari A, Roloff T, Seth-Smith HMB, Schweitzer M, Brunner M, Leuzinger K, Søgaard KK, Gensch A, Tschudin-Sutter S, Fuchs S, Bielicki J, Pargger H, Siegemund M, Nickel CH, Bingisser R, Osthoff M, Bassetti S, Schneider-Sliwa R, Battegay M, Hirsch HH, Egli A. 2021. SARS-CoV-2 outbreak in a tri-national urban area is dominated by a B.1 lineage variant linked to a mass gathering event. PLoS Pathog 17:e1009374. 10.1371/journal.ppat.1009374. - DOI - PMC - PubMed
1. Brüningk SC, Klatt J, Stange M, Mari A, Brunner M, Roloff T-C, Seth-Smith HMB, Schweitzer M, Leuzinger K, Søgaard KK, Torres DA, Gensch A, Schlotterbeck A-K, Nickel CH, Ritz N, Heininger U, Bielicki J, Rentsch K, Fuchs S, Bingisser R, Siegemund M, Pargger H, Ciardo D, Dubuis O, Buser A, Tschudin-Sutter S, Battegay M, Schneider-Sliwa R, Borgwardt KM, Hirsch HH, Egli A. 2020. Determinants of SARS-CoV-2 transmission to guide vaccination strategy in a city. medRxiv 10.1101/2020.12.15.20248130. - DOI - PMC - PubMed

Publication types

Actions

MeSH terms

Actions
Actions
Actions
Actions
Actions
Actions

LinkOut - more resources

Full Text Sources
Medical
- MedlinePlus Health Information
Miscellaneous
- NCI CPTAC Assay Portal

Save citation to file

Email citation

Add to Collections

Add to My Bibliography

Your saved search

Create a file for external citation management software

Your RSS Feed

External Quality Assessment of SARS-CoV-2 Sequencing: an ESGMD-SSM Pilot Trial across 15 European Laboratories

Affiliations

External Quality Assessment of SARS-CoV-2 Sequencing: an ESGMD-SSM Pilot Trial across 15 European Laboratories

Authors

Affiliations

Abstract

Conflict of interest statement

Figures

References

Publication types

MeSH terms

LinkOut - more resources

Full Text Sources

Medical

Miscellaneous