Critical Assessment of Metagenome Interpretation – a benchmark of computational metagenomics software
Alexander Sczyrba
(1, 2)
,
Peter Hofmann
(3, 4, 5)
,
Peter Belmann
(1, 4, 5)
,
David Koslicki
(6)
,
Stefan Janssen
(4, 5, 7, 8)
,
Johannes Dröge
(3, 4, 5)
,
Ivan Gregor
(3, 9)
,
Stephan Majda
(3, 10)
,
Jessika Fiedler
(3, 4, 5)
,
Eik Dahms
(3, 4, 5)
,
Andreas Bremges
(1, 2, 11, 4, 5)
,
Adrian Fritz
(4, 5)
,
Ruben Garrido-Oter
(3, 12, 13)
,
Tue Sparholt Jørgensen
(14, 15, 16)
,
Nicole Shapiro
(17)
,
Philip D Blood
(18)
,
Alexey Gurevich
(19)
,
Yang Bai
(12, 20)
,
Dmitrij Turaev
(21)
,
Matthew Z Demaere
(22)
,
Rayan Chikhi
(23, 24)
,
Niranjan Nagarajan
(25)
,
Christopher Quince
(26)
,
Lars Hestbjerg Hansen
(14)
,
Søren J Sørensen
(15)
,
Burton K H Chia
(25)
,
Bertrand Denis
(25)
,
Jeff L Froula
(17)
,
Zhong Wang
(17)
,
Robert Egan
(17)
,
Dongwan Don Kang
(17)
,
Jeffrey J Cook
(27)
,
Charles Deltel
(28)
,
Michael Beckstette
(29)
,
Claire Lemaitre
(28)
,
Peter Peterlongo
(28)
,
Guillaume Rizk
(28)
,
Dominique Lavenier
(28)
,
Yu-Wei Wu
(30, 31)
,
Steven W Singer
(30, 32)
,
Chirag Jain
(33)
,
Marc Strous
(34)
,
Heiner Klingenberg
(35)
,
Peter Meinicke
(35)
,
Michael D Barton
(17)
,
Thomas Lingner
(36)
,
Hsin-Hung Lin
(37)
,
Yu-Chieh Liao
(37)
,
Genivaldo Gueiros Z Silva
(38)
,
Daniel A Cuevas
(38)
,
Robert A Edwards
(38)
,
Surya Saha
(39)
,
Vitor C Piro
(40, 41)
,
Bernhard y Renard
(40)
,
Mihai Pop
(42)
,
Hans-Peter Klenk
(43)
,
Markus Göker
(44)
,
Nikos C Kyrpides
(17, 45)
,
Tanja Woyke
(17)
,
Julia A Vorholt
(46, 47)
,
Paul Schulze-Lefert
(12, 13)
,
Edward M Rubin
(17)
,
Aaron E Darling
(22)
,
Thomas Rattei
(21)
,
Alice C Mchardy
(3, 4, 5, 13)
1
CeBiTec -
Center for Biotechnology
2 Technische Fakultät, Universität Bielefeld
3 Algorithmische Bioinformatik [Düsseldorf]
4 Computational Biology of Infection Research [Braunschweig]
5 BRICS - Braunschweig Integrated Centre of Systems Biology [Braunschweig]
6 Department of Mathematics [Corvallis, Oregon]
7 CSE - UC San Diego - Department of Computer Science and Engineering [Univ California San Diego]
8 UC San Diego - Department of Pediatrics [Univ California San Diego]
9 Max Planck Institute for Informatics [Saarbrücken]
10 Faculty of Biology [Essen]
11 DZIF - German Center for Infection Research - partner site Hannover-Braunschweig
12 Department of Plant Microbe Interactions
13 CEPLAS - Cluster of Excellence on Plant Sciences
14 ENVS - Department of Environmental Science [Roskilde]
15 Section of Microbiology [Copenhagen]
16 Department of Science and Environment [Roskilde]
17 DOE Joint Genome Institute [Walnut Creek]
18 PSC - Pittsburgh Supercomputing Center
19 Center for Algorithmic Biotechnology [Saint Petersburg]
20 CEPAMS - Centre of Excellence for Plant and Microbial Sciences
21 Department of Microbiology and Ecosystem Science [Vienna]
22 iThree Institute
23 BONSAI - Bioinformatics and Sequence Analysis
24 CRIStAL - Centre de Recherche en Informatique, Signal et Automatique de Lille - UMR 9189
25 Department of Computational and Systems Biology [Singapore]
26 Department of Microbiology and Infection [Coventry]
27 Intel Corporation [Hillsboro]
28 GenScale - Scalable, Optimized and Parallel Algorithms for Genomics
29 Department of Molecular Infection Biology [Braunschweig]
30 Joint BioEnergy Institute [Emeryville]
31 Graduate Institute of Biomedical Informatics [Taipei]
32 Biological Systems and Engineering [LBNL Berkeley]
33 Max planck Institute for Biology of Ageing [Cologne]
34 Energy Engineering and Geomicrobiology [Calgary]
35 Institute of Microbiology and Genetics [Göttingen]
36 UMG - University Medical Center Göttingen
37 Institute of Population Health Sciences [Taiwan]
38 SDSU - San Diego State University
39 Boyce Thompson Institute [Ithaca]
40 RKI - Robert Koch Institute [Berlin]
41 Ministry of Education [Brazil]
42 CBCB - Center for Bioinformatics and Computational Biology [Maryland]
43 School of Biology [Newcastle upon Tyne]
44 DSMZ - Leibniz-Institut DSMZ-Deutsche Sammlung von Mikroorganismen und Zellkulturen GmbH / Leibniz Institute DSMZ-German Collection of Microorganisms and Cell Cultures
45 biological sciences department [Jeddah]
46 Institute of Microbiology [Zurich]
47 ETH Zürich - Eidgenössische Technische Hochschule - Swiss Federal Institute of Technology [Zürich]
2 Technische Fakultät, Universität Bielefeld
3 Algorithmische Bioinformatik [Düsseldorf]
4 Computational Biology of Infection Research [Braunschweig]
5 BRICS - Braunschweig Integrated Centre of Systems Biology [Braunschweig]
6 Department of Mathematics [Corvallis, Oregon]
7 CSE - UC San Diego - Department of Computer Science and Engineering [Univ California San Diego]
8 UC San Diego - Department of Pediatrics [Univ California San Diego]
9 Max Planck Institute for Informatics [Saarbrücken]
10 Faculty of Biology [Essen]
11 DZIF - German Center for Infection Research - partner site Hannover-Braunschweig
12 Department of Plant Microbe Interactions
13 CEPLAS - Cluster of Excellence on Plant Sciences
14 ENVS - Department of Environmental Science [Roskilde]
15 Section of Microbiology [Copenhagen]
16 Department of Science and Environment [Roskilde]
17 DOE Joint Genome Institute [Walnut Creek]
18 PSC - Pittsburgh Supercomputing Center
19 Center for Algorithmic Biotechnology [Saint Petersburg]
20 CEPAMS - Centre of Excellence for Plant and Microbial Sciences
21 Department of Microbiology and Ecosystem Science [Vienna]
22 iThree Institute
23 BONSAI - Bioinformatics and Sequence Analysis
24 CRIStAL - Centre de Recherche en Informatique, Signal et Automatique de Lille - UMR 9189
25 Department of Computational and Systems Biology [Singapore]
26 Department of Microbiology and Infection [Coventry]
27 Intel Corporation [Hillsboro]
28 GenScale - Scalable, Optimized and Parallel Algorithms for Genomics
29 Department of Molecular Infection Biology [Braunschweig]
30 Joint BioEnergy Institute [Emeryville]
31 Graduate Institute of Biomedical Informatics [Taipei]
32 Biological Systems and Engineering [LBNL Berkeley]
33 Max planck Institute for Biology of Ageing [Cologne]
34 Energy Engineering and Geomicrobiology [Calgary]
35 Institute of Microbiology and Genetics [Göttingen]
36 UMG - University Medical Center Göttingen
37 Institute of Population Health Sciences [Taiwan]
38 SDSU - San Diego State University
39 Boyce Thompson Institute [Ithaca]
40 RKI - Robert Koch Institute [Berlin]
41 Ministry of Education [Brazil]
42 CBCB - Center for Bioinformatics and Computational Biology [Maryland]
43 School of Biology [Newcastle upon Tyne]
44 DSMZ - Leibniz-Institut DSMZ-Deutsche Sammlung von Mikroorganismen und Zellkulturen GmbH / Leibniz Institute DSMZ-German Collection of Microorganisms and Cell Cultures
45 biological sciences department [Jeddah]
46 Institute of Microbiology [Zurich]
47 ETH Zürich - Eidgenössische Technische Hochschule - Swiss Federal Institute of Technology [Zürich]
Alexander Sczyrba
- Fonction : Auteur correspondant
- PersonId : 784780
- ORCID : 0000-0002-4405-3847
Stephan Majda
- Fonction : Auteur
Rayan Chikhi
- Fonction : Auteur
- PersonId : 14839
- IdHAL : rayan-chikhi
- ORCID : 0000-0003-1099-8735
- IdRef : 16546769X
Christopher Quince
- Fonction : Auteur
- PersonId : 776169
- ORCID : 0000-0003-1884-8440
Zhong Wang
- Fonction : Auteur
- PersonId : 759342
- ORCID : 0000-0002-6144-4022
Claire Lemaitre
- Fonction : Auteur
- PersonId : 914
- IdHAL : claire-lemaitre
- ORCID : 0000-0001-8675-170X
- IdRef : 132940418
Dominique Lavenier
- Fonction : Auteur
- PersonId : 1401
- IdHAL : dominique-lavenier
- ORCID : 0000-0003-2557-680X
Tanja Woyke
- Fonction : Auteur
- PersonId : 763506
- ORCID : 0000-0002-9485-5637
Paul Schulze-Lefert
- Fonction : Auteur
- PersonId : 969536
Thomas Rattei
- Fonction : Auteur
- PersonId : 772530
- ORCID : 0000-0002-0592-7791
Résumé
In metagenome analysis, computational methods for assembly, taxonomic profilingand binning are key components facilitating downstream biological datainterpretation. However, a lack of consensus about benchmarking datasets andevaluation metrics complicates proper performance assessment. The CriticalAssessment of Metagenome Interpretation (CAMI) challenge has engaged the globaldeveloper community to benchmark their programs on datasets of unprecedentedcomplexity and realism. Benchmark metagenomes were generated from newlysequenced ~700 microorganisms and ~600 novel viruses and plasmids, includinggenomes with varying degrees of relatedness to each other and to publicly availableones and representing common experimental setups. Across all datasets, assemblyand genome binning programs performed well for species represented by individualgenomes, while performance was substantially affected by the presence of relatedstrains. Taxonomic profiling and binning programs were proficient at high taxonomicranks, with a notable performance decrease below the family level. Parametersettings substantially impacted performances, underscoring the importance ofprogram reproducibility. While highlighting current challenges in computationalmetagenomics, the CAMI results provide a roadmap for software selection to answerspecific research questions.
Domaines
Bio-informatique [q-bio.QM]| Origine | Fichiers produits par l'(les) auteur(s) |
|---|
Loading...