FAIR data pipeline: provenance-driven data management for traceable scientific workflows
Sonia Natalie Mitchell
,
Andrew Lahiff
,
Nathan Cummings
,
Jonathan Hollocombe
,
Bram Boskamp
,
Ryan Field
,
Dennis Reddyhoff
,
Kristian Zarebski
,
Antony Wilson
,
Bruno Viola
,
Martin Burke
,
Blair Archibald
,
Paul Bessell
,
Richard Blackwell
,
Lisa Boden
,
Alys Brett
,
Sam Brett
,
Ruth Dundas
,
Jessica Enright
,
Alejandra Gonzalez-Beltran
,
Claire Harris
,
Ian Hinder
,
Christopher David Hughes
,
Martin Knight
,
Vino Mano
,
Ciaran Mcmonagle
,
Dominic Mellor
,
Sibylle Mohr
,
Glenn Marion
,
Louise Matthews
,
Iain Mckendrick
,
Christopher Mark Pooley
,
Thibaud Porphyre
(1)
,
Aaron Reeves
,
Edward Townsend
,
Robert Turner
,
Jeremy Walton
,
Richard Reeve
Sonia Natalie Mitchell
- Function : Author
- PersonId : 1156320
- ORCID : 0000-0003-1536-2066
Andrew Lahiff
- Function : Author
- PersonId : 1156321
- ORCID : 0000-0002-2785-4116
Nathan Cummings
- Function : Author
- PersonId : 1156322
- ORCID : 0000-0003-4359-6337
Jonathan Hollocombe
- Function : Author
- PersonId : 1156323
- ORCID : 0000-0001-7159-158X
Bram Boskamp
- Function : Author
- PersonId : 1156324
- ORCID : 0000-0002-5446-3209
Ryan Field
- Function : Author
- PersonId : 1156325
- ORCID : 0000-0002-4424-9890
Dennis Reddyhoff
- Function : Author
- PersonId : 1156326
- ORCID : 0000-0002-4971-2346
Kristian Zarebski
- Function : Author
- PersonId : 1156327
- ORCID : 0000-0002-6773-1049
Antony Wilson
- Function : Author
- PersonId : 1156328
- ORCID : 0000-0001-7336-4823
Bruno Viola
- Function : Author
- PersonId : 1156329
- ORCID : 0000-0001-5406-5860
Martin Burke
- Function : Author
- PersonId : 1156330
- ORCID : 0000-0002-5540-9087
Blair Archibald
- Function : Author
- PersonId : 1156331
- ORCID : 0000-0003-3699-6658
Paul Bessell
- Function : Author
- PersonId : 1156332
- ORCID : 0000-0002-7901-969X
Richard Blackwell
- Function : Author
Lisa Boden
- Function : Author
- PersonId : 1156333
- ORCID : 0000-0002-9317-4741
Alys Brett
- Function : Author
- PersonId : 1156334
- ORCID : 0000-0002-5789-0572
Sam Brett
- Function : Author
Ruth Dundas
- Function : Author
- PersonId : 1156335
- ORCID : 0000-0002-3836-4286
Jessica Enright
- Function : Author
- PersonId : 1156336
- ORCID : 0000-0002-0266-3292
Alejandra Gonzalez-Beltran
- Function : Author
- PersonId : 775719
- ORCID : 0000-0003-3499-8262
Claire Harris
- Function : Author
- PersonId : 1156337
- ORCID : 0000-0003-0852-2340
Ian Hinder
- Function : Author
- PersonId : 1156338
- ORCID : 0000-0003-3548-9101
Christopher David Hughes
- Function : Author
Martin Knight
- Function : Author
- PersonId : 1156339
- ORCID : 0000-0002-3812-4397
Vino Mano
- Function : Author
Ciaran Mcmonagle
- Function : Author
- PersonId : 1156340
- ORCID : 0000-0002-1325-2413
Dominic Mellor
- Function : Author
- PersonId : 1156341
- ORCID : 0000-0001-7141-652X
Sibylle Mohr
- Function : Author
- PersonId : 1156342
- ORCID : 0000-0002-9089-6327
Glenn Marion
- Function : Author
- PersonId : 1155031
- ORCID : 0000-0002-0454-9338
Louise Matthews
- Function : Author
- PersonId : 1156343
- ORCID : 0000-0003-4420-8367
Iain Mckendrick
- Function : Author
- PersonId : 1156344
- ORCID : 0000-0002-7529-0749
Christopher Mark Pooley
- Function : Author
- PersonId : 1155030
- ORCID : 0000-0002-8779-4477
Thibaud Porphyre
- Function : Author
- PersonId : 1155014
- IdHAL : thibaud-porphyre
- ORCID : 0000-0002-8552-9899
Aaron Reeves
- Function : Author
- PersonId : 1156345
- ORCID : 0000-0001-6387-7481
Edward Townsend
- Function : Author
Robert Turner
- Function : Author
- PersonId : 1156346
- ORCID : 0000-0002-1353-1404
Jeremy Walton
- Function : Author
- PersonId : 816106
- ORCID : 0000-0001-7372-178X
Richard Reeve
- Function : Author
- PersonId : 1155042
- ORCID : 0000-0003-2589-8091
Abstract
Modern epidemiological analyses to understand and combat the spread of disease depend critically on access to, and use of, data. Rapidly evolving data, such as data streams changing during a disease outbreak, are particularly challenging. Data management is further complicated by data being imprecisely identified when used. Public trust in policy decisions resulting from such analyses is easily damaged and is often low, with cynicism arising where claims of ‘following the science’ are made without accompanying evidence. Tracing the provenance of such decisions back through open software to primary data would clarify this evidence, enhancing the transparency of the decision-making process. Here, we demonstrate a Findable, Accessible, Interoperable and Reusable (FAIR) data pipeline. Although developed during the COVID-19 pandemic, it allows easy annotation of any data as they are consumed by analyses, or conversely traces the provenance of scientific outputs back through the analytical or modelling source code to primary data. Such a tool provides a mechanism for the public, and fellow scientists, to better assess scientific evidence by inspecting its provenance, while allowing scientists to support policymakers in openly justifying their decisions. We believe that such tools should be promoted for use across all areas of policy-facing research. This article is part of the theme issue ‘Technical challenges of modelling real-life epidemics and examples of overcoming these’.