BigBIO: A Framework for Data-Centric Biomedical Natural Language Processing
Jason Alan Fries
(1)
,
Leon Weber
(2, 3)
,
Natasha Seelam
,
Gabriel Altay
(4)
,
Debajyoti Datta
(5)
,
Samuele Garda
(2)
,
Myungsun Kang
(6)
,
Ruisi Su
(7)
,
Wojciech Kusa
(8)
,
Samuel Cahyawijaya
(9)
,
Fabio Barth
(2)
,
Simon Ott
(10)
,
Matthias Samwald
(10)
,
Stephen Bach
(8)
,
Stella Biderman
(11, 12)
,
Mario Sänger
(2)
,
Bo Wang
,
Alison Callahan
(1)
,
Daniel León Periñán
,
Théo Gigant
(7)
,
Patrick Haller
(2)
,
Jenny Chim
,
Jose David Posada
,
John Michael Giorgi
,
Karthik Rangasai Sivaraman
,
Marc Pàmies
,
Marianna Nezhurina
,
Robert Martin
(2)
,
Michael Cullan
,
Moritz Freidank
,
Nathan Dahlberg
(7)
,
Shubhanshu Mishra
,
Shamik Bose
(7)
,
Nicholas Michio Broad
,
Yanis Labrak
(13)
,
Shlok S Deshmukh
,
Sid Kiblawi
,
Ayush Singh
(7)
,
Minh Chien Vu
,
Trishala Neeraj
,
Jonas Golde
(2)
,
Albert Villanova del Moral
,
Benjamin Beilharz
1
Stanford University
2 HU Berlin - Humboldt-Universität zu Berlin = Humboldt University of Berlin = Université Humboldt de Berlin
3 MDC - Max Delbrück Center for Molecular Medicine [Berlin]
4 Tempus Labs Inc.
5 University of Virginia
6 Immuneering Corporation
7 BigScience Project
8 Brown University
9 HKUST - The Hong Kong University of Science and Technology
10 Universität Wien = University of Vienna
11 EleutherAI
12 Booz Allen Hamilton
13 LIA - Laboratoire Informatique d'Avignon
2 HU Berlin - Humboldt-Universität zu Berlin = Humboldt University of Berlin = Université Humboldt de Berlin
3 MDC - Max Delbrück Center for Molecular Medicine [Berlin]
4 Tempus Labs Inc.
5 University of Virginia
6 Immuneering Corporation
7 BigScience Project
8 Brown University
9 HKUST - The Hong Kong University of Science and Technology
10 Universität Wien = University of Vienna
11 EleutherAI
12 Booz Allen Hamilton
13 LIA - Laboratoire Informatique d'Avignon
Natasha Seelam
- Fonction : Auteur
Bo Wang
- Fonction : Auteur
Daniel León Periñán
- Fonction : Auteur
Jenny Chim
- Fonction : Auteur
Jose David Posada
- Fonction : Auteur
John Michael Giorgi
- Fonction : Auteur
Karthik Rangasai Sivaraman
- Fonction : Auteur
Marc Pàmies
- Fonction : Auteur
Marianna Nezhurina
- Fonction : Auteur
Michael Cullan
- Fonction : Auteur
Moritz Freidank
- Fonction : Auteur
Shubhanshu Mishra
- Fonction : Auteur
Nicholas Michio Broad
- Fonction : Auteur
Yanis Labrak
- Fonction : Auteur
- PersonId : 750624
- IdHAL : yanis-labrak
- ORCID : 0000-0003-1072-3862
Shlok S Deshmukh
- Fonction : Auteur
Sid Kiblawi
- Fonction : Auteur
Minh Chien Vu
- Fonction : Auteur
Trishala Neeraj
- Fonction : Auteur
Albert Villanova del Moral
- Fonction : Auteur
Benjamin Beilharz
- Fonction : Auteur
Résumé
Training and evaluating language models increasingly requires the construction of meta-datasets --diverse collections of curated data with clear provenance. Natural language prompting has recently lead to improved zero-shot generalization by transforming existing, supervised datasets into a diversity of novel pretraining tasks, highlighting the benefits of meta-dataset curation. While successful in general-domain text, translating these data-centric approaches to biomedical language modeling remains challenging, as labeled biomedical datasets are significantly underrepresented in popular data hubs. To address this challenge, we introduce BigBIO a community library of 126+ biomedical NLP datasets, currently covering 12 task categories and 10+ languages. BigBIO facilitates reproducible meta-dataset curation via programmatic access to datasets and their metadata, and is compatible with current platforms for prompt engineering and end-to-end few/zero shot language model evaluation. We discuss our process for task schema harmonization, data auditing, contribution guidelines, and outline two illustrative use cases: zero-shot evaluation of biomedical prompts and large-scale, multi-task learning. BigBIO is an ongoing community effort and is available at https://github.com/bigscience-workshop/biomedical