Critical Assessment of Metagenome Interpretation: the second round of challenges
Fernando Meyer
(1)
,
Adrian Fritz
(1)
,
Zhi-Luo Deng
(1)
,
David Koslicki
(2)
,
Till Robin Lesker
(3)
,
Alexey Gurevich
(4)
,
Gary Robertson
(1)
,
Mohammed Alser
(5)
,
Dmitry Antipov
(6)
,
Francesco Beghini
(7)
,
Denis Bertrand
(8)
,
Jaqueline Brito
(9)
,
C. Titus Brown
(10)
,
Jan Buchmann
(11)
,
Aydin Buluç
(12)
,
Bo Chen
(12)
,
Rayan Chikhi
(13)
,
Philip Clausen
(14)
,
Alexandru Cristian
(15)
,
Piotr Wojciech Dabrowski
(16)
,
Aaron Darling
(17)
,
Rob Egan
(18)
,
Eleazar Eskin
(19)
,
Evangelos Georganas
(20)
,
Eugene Goltsman
(18)
,
Melissa Gray
(15)
,
Lars Hestbjerg Hansen
(21)
,
Steven Hofmeyr
(12)
,
Pingqin Huang
(22)
,
Luiz Irber
(10)
,
Huijue Jia
(23)
,
Tue Sparholt Jørgensen
(24)
,
Silas Kieser
(25)
,
Terje Klemetsen
(26)
,
Axel Kola
(27)
,
Mikhail Kolmogorov
(28)
,
Anton Korobeynikov
(4)
,
Jason Kwan
(29)
,
Nathan Lapierre
(19)
,
Claire Lemaitre
(30)
,
Chenhao Li
(8)
,
Antoine Limasset
(31, 32)
,
Fabio Malcher-Miranda
(33)
,
Serghei Mangul
(9)
,
Vanessa Marcelino
(34)
,
Camille Marchet
(32)
,
Pierre Marijon
(35)
,
Dmitry Meleshko
(6)
,
Daniel Mende
(36)
,
Alessio Milanese
(37)
,
Niranjan Nagarajan
(8)
,
Jakob Nissen
(38)
,
Sergey Nurk
(39)
,
Leonid Oliker
(12)
,
Lucas Paoli
(40)
,
Vitor Piro
(33)
,
Jacob Porter
(41)
,
Simon Rasmussen
(42)
,
Evan Rees
(29)
,
Knut Reinert
(43)
,
Bernhard Renard
(44)
,
Espen Mikal Robertsen
(26)
,
Gail Rosen
(15)
,
Hans-Joachim Ruscheweyh
(40)
,
Varuni Sarwal
(19)
,
Nicola Segata
(7)
,
Enrico Seiler
(43)
,
Lizhen Shi
(45)
,
Fengzhu Sun
(9)
,
Shinichi Sunagawa
(43)
,
Søren Johannes Sørensen
(42)
,
Ashleigh Thomas
(18)
,
Chengxuan Tong
(8)
,
Mirko Trajkovski
(25)
,
Julien Tremblay
(46)
,
Gherman Uritskiy
(47)
,
Riccardo Vicedomini
(13)
,
Zhengyang Wang
(22)
,
Ziye Wang
(22)
,
Zhong Wang
(12)
,
Andrew Warren
(41)
,
Nils Peder Willassen
(26)
,
Katherine Yelick
(12)
,
Ronghui You
(22)
,
Georg Zeller
(37)
,
Zhengqiao Zhao
(15)
,
Shanfeng Zhu
(22)
,
Jie Zhu
(23)
,
Ruben Garrido-Oter
(48)
,
Petra Gastmeier
(27)
,
Stephane Hacquard
(48)
,
Susanne Häußler
(49)
,
Ariane Khaledi
(49)
,
Friederike Maechler
(27)
,
Fantin Mesny
(48)
,
Simona Radutoiu
(50)
,
Paul Schulze-Lefert
(48)
,
Nathiana Smit
(49)
,
Till Strowig
(49)
,
Andreas Bremges
(49)
,
Alexander Sczyrba
(51)
,
Alice Carolyn Mchardy
(1)
,
Pierre Peterlongo
(30)
1
BRICS -
Braunschweig Integrated Centre of Systems Biology [Braunschweig]
2 Penn State - Pennsylvania State University
3 DZIF - German Center for Infection Research - partner site Hannover-Braunschweig
4 SPBU - Saint Petersburg State University
5 D-ITET - Department of Information Technology and Electrical Engineering [Zürich]
6 Center for Algorithmic Biotechnology [Saint Petersburg]
7 CIBIO - Centre for Integrative Biology (CIBIO), University of Trento
8 GIS - Genome Institute of Singapore
9 USC - University of Southern California
10 UC Davis - University of California [Davis]
11 Heinrich Heine Universität Düsseldorf = Heinrich Heine University [Düsseldorf]
12 LBNL - Lawrence Berkeley National Laboratory [Berkeley]
13 IP - Institut Pasteur [Paris]
14 Forside - National Food Institute [Lyngby]
15 Drexel University
16 RKI - Robert Koch Institute [Berlin]
17 UTS - University of Technology Sydney
18 DOE Joint Genome Institute [Walnut Creek]
19 UCLA - University of California [Los Angeles]
20 Intel Corporation [Santa Clara]
21 Department of Plant and Environmental Sciences [Frederiksberg]
22 Fudan University [Shanghai]
23 BGI - Beijing Genomics Institute [Shenzhen]
24 Novo Nordisk Foundation Center for Biosustainability
25 UNIGE - Université de Genève = University of Geneva
26 UiT - The Arctic University of Norway [Tromsø, Norway]
27 Charité - UniversitätsMedizin = Berlin University Medicine
28 UC San Diego - University of California [San Diego]
29 University of Wisconsin-Madison
30 GenScale - Scalable, Optimized and Parallel Algorithms for Genomics
31 CNRS - Centre National de la Recherche Scientifique
32 CRIStAL - Centre de Recherche en Informatique, Signal et Automatique de Lille - UMR 9189
33 Hasso Plattner Institute [Potsdam, Germany]
34 The University of Sydney
35 Inria Lille - Nord Europe
36 Amsterdam UMC - Amsterdam University Medical Centers
37 Structural and Computational Biology
38 DTU Electrical Engineering [Lyngby]
39 NIH - National Institutes of Health [Bethesda, MD, USA]
40 D-BIOL - Department of Biology [ETH Zürich]
41 University of Virginia
42 ITU - IT University of Copenhagen
43 Freie Universität Berlin
44 University of Potsdam = Universität Potsdam
45 FIU - Florida International University [Miami]
46 NRC - National Research Council of Canada
47 Phase Genomics [Seattle]
48 MPIPZ - Max Planck Institute for Plant Breeding Research
49 HZI - Helmholtz Centre for Infection Research
50 Aarhus University [Aarhus]
51 CeBiTec - Center for Biotechnology
2 Penn State - Pennsylvania State University
3 DZIF - German Center for Infection Research - partner site Hannover-Braunschweig
4 SPBU - Saint Petersburg State University
5 D-ITET - Department of Information Technology and Electrical Engineering [Zürich]
6 Center for Algorithmic Biotechnology [Saint Petersburg]
7 CIBIO - Centre for Integrative Biology (CIBIO), University of Trento
8 GIS - Genome Institute of Singapore
9 USC - University of Southern California
10 UC Davis - University of California [Davis]
11 Heinrich Heine Universität Düsseldorf = Heinrich Heine University [Düsseldorf]
12 LBNL - Lawrence Berkeley National Laboratory [Berkeley]
13 IP - Institut Pasteur [Paris]
14 Forside - National Food Institute [Lyngby]
15 Drexel University
16 RKI - Robert Koch Institute [Berlin]
17 UTS - University of Technology Sydney
18 DOE Joint Genome Institute [Walnut Creek]
19 UCLA - University of California [Los Angeles]
20 Intel Corporation [Santa Clara]
21 Department of Plant and Environmental Sciences [Frederiksberg]
22 Fudan University [Shanghai]
23 BGI - Beijing Genomics Institute [Shenzhen]
24 Novo Nordisk Foundation Center for Biosustainability
25 UNIGE - Université de Genève = University of Geneva
26 UiT - The Arctic University of Norway [Tromsø, Norway]
27 Charité - UniversitätsMedizin = Berlin University Medicine
28 UC San Diego - University of California [San Diego]
29 University of Wisconsin-Madison
30 GenScale - Scalable, Optimized and Parallel Algorithms for Genomics
31 CNRS - Centre National de la Recherche Scientifique
32 CRIStAL - Centre de Recherche en Informatique, Signal et Automatique de Lille - UMR 9189
33 Hasso Plattner Institute [Potsdam, Germany]
34 The University of Sydney
35 Inria Lille - Nord Europe
36 Amsterdam UMC - Amsterdam University Medical Centers
37 Structural and Computational Biology
38 DTU Electrical Engineering [Lyngby]
39 NIH - National Institutes of Health [Bethesda, MD, USA]
40 D-BIOL - Department of Biology [ETH Zürich]
41 University of Virginia
42 ITU - IT University of Copenhagen
43 Freie Universität Berlin
44 University of Potsdam = Universität Potsdam
45 FIU - Florida International University [Miami]
46 NRC - National Research Council of Canada
47 Phase Genomics [Seattle]
48 MPIPZ - Max Planck Institute for Plant Breeding Research
49 HZI - Helmholtz Centre for Infection Research
50 Aarhus University [Aarhus]
51 CeBiTec - Center for Biotechnology
Fernando Meyer
- Fonction : Auteur
- PersonId : 1179914
- ORCID : 0000-0002-0901-3815
Adrian Fritz
- Fonction : Auteur
- PersonId : 1179915
- ORCID : 0000-0002-9853-5577
David Koslicki
- Fonction : Auteur
- PersonId : 1179916
- ORCID : 0000-0002-0640-954X
Alexey Gurevich
- Fonction : Auteur
- PersonId : 1179917
- ORCID : 0000-0002-5855-3519
Francesco Beghini
- Fonction : Auteur
- PersonId : 1179918
- ORCID : 0000-0002-8105-9607
C. Titus Brown
- Fonction : Auteur
- PersonId : 1179919
- ORCID : 0000-0001-6001-2677
Jan Buchmann
- Fonction : Auteur
- PersonId : 1179920
- ORCID : 0000-0002-6842-1229
Rayan Chikhi
- Fonction : Auteur
- PersonId : 14839
- IdHAL : rayan-chikhi
- ORCID : 0000-0003-1099-8735
- IdRef : 16546769X
Philip Clausen
- Fonction : Auteur
- PersonId : 1179921
- ORCID : 0000-0002-8197-7520
Piotr Wojciech Dabrowski
- Fonction : Auteur
- PersonId : 822730
- ORCID : 0000-0003-4893-805X
Aaron Darling
- Fonction : Auteur
- PersonId : 763113
- ORCID : 0000-0003-2397-7925
Tue Sparholt Jørgensen
- Fonction : Auteur
- PersonId : 1179922
- ORCID : 0000-0002-2437-7086
Silas Kieser
- Fonction : Auteur
- PersonId : 1179923
- ORCID : 0000-0002-5935-9734
Anton Korobeynikov
- Fonction : Auteur
- PersonId : 772217
- ORCID : 0000-0002-2937-9259
Claire Lemaitre
- Fonction : Auteur
- PersonId : 914
- IdHAL : claire-lemaitre
- ORCID : 0000-0001-8675-170X
- IdRef : 132940418
Antoine Limasset
- Fonction : Auteur
- PersonId : 180632
- IdHAL : antoine-limasset
- ORCID : 0000-0002-0669-4141
- IdRef : 223503908
Fabio Malcher-Miranda
- Fonction : Auteur
- PersonId : 1179924
- ORCID : 0000-0002-6823-5995
Alessio Milanese
- Fonction : Auteur
- PersonId : 1179925
- ORCID : 0000-0002-7050-2239
Niranjan Nagarajan
- Fonction : Auteur
- PersonId : 1179926
- ORCID : 0000-0003-0850-5604
Vitor Piro
- Fonction : Auteur
- PersonId : 1179927
- ORCID : 0000-0003-1330-1286
Simon Rasmussen
- Fonction : Auteur
- PersonId : 1179928
- ORCID : 0000-0001-6323-9041
Evan Rees
- Fonction : Auteur
- PersonId : 1179929
- ORCID : 0000-0001-8405-9760
Bernhard Renard
- Fonction : Auteur
- PersonId : 806266
- ORCID : 0000-0003-4589-9809
Nicola Segata
- Fonction : Auteur
- PersonId : 793997
- ORCID : 0000-0002-1583-5794
Shinichi Sunagawa
- Fonction : Auteur
- PersonId : 1166109
- ORCID : 0000-0003-3065-0314
Søren Johannes Sørensen
- Fonction : Auteur
- PersonId : 1179930
- ORCID : 0000-0001-6227-9906
Ashleigh Thomas
- Fonction : Auteur
- PersonId : 1179931
- ORCID : 0000-0001-7422-5649
Mirko Trajkovski
- Fonction : Auteur
- PersonId : 1179932
- ORCID : 0000-0002-3152-9551
Julien Tremblay
- Fonction : Auteur
- PersonId : 1297736
- ORCID : 0000-0002-6085-3481
Riccardo Vicedomini
- Fonction : Auteur
- PersonId : 750807
- IdHAL : rvicedomini
- ORCID : 0000-0002-7706-0998
Zhengyang Wang
- Fonction : Auteur
- PersonId : 1179933
- ORCID : 0000-0003-0118-1010
Georg Zeller
- Fonction : Auteur
- PersonId : 1179934
- ORCID : 0000-0003-1429-7485
Jie Zhu
- Fonction : Auteur
- PersonId : 1179936
- ORCID : 0000-0002-5789-6296
Ruben Garrido-Oter
- Fonction : Auteur
- PersonId : 1179937
- ORCID : 0000-0003-1769-892X
Stephane Hacquard
- Fonction : Auteur
- PersonId : 774787
- ORCID : 0000-0003-2293-3525
- IdRef : 151029717
Susanne Häußler
- Fonction : Auteur
- PersonId : 1172607
- ORCID : 0000-0001-6141-9102
Fantin Mesny
- Fonction : Auteur
- PersonId : 819289
- ORCID : 0000-0002-3044-1398
Paul Schulze-Lefert
- Fonction : Auteur
- PersonId : 1179938
- ORCID : 0000-0002-8978-1717
Alexander Sczyrba
- Fonction : Auteur
- PersonId : 784780
- ORCID : 0000-0002-4405-3847
Alice Carolyn Mchardy
- Fonction : Auteur
- PersonId : 1179939
- ORCID : 0000-0003-2370-3430
Pierre Peterlongo
- Fonction : Auteur
- PersonId : 171998
- IdHAL : pierre-peterlongo
- ORCID : 0000-0003-0776-6407
- IdRef : 12482062X
Résumé
Abstract Evaluating metagenomic software is key for optimizing metagenome interpretation and focus of the Initiative for the Critical Assessment of Metagenome Interpretation (CAMI). The CAMI II challenge engaged the community to assess methods on realistic and complex datasets with long- and short-read sequences, created computationally from around 1,700 new and known genomes, as well as 600 new plasmids and viruses. Here we analyze 5,002 results by 76 program versions. Substantial improvements were seen in assembly, some due to long-read data. Related strains still were challenging for assembly and genome recovery through binning, as was assembly quality for the latter. Profilers markedly matured, with taxon profilers and binners excelling at higher bacterial ranks, but underperforming for viruses and Archaea. Clinical pathogen detection results revealed a need to improve reproducibility. Runtime and memory usage analyses identified efficient programs, including top performers with other metrics. The results identify challenges and guide researchers in selecting methods for analyses.
Domaines
Bio-informatique [q-bio.QM]| Origine | Publication financée par une institution |
|---|---|
| Licence |
