AI evaluation campaigns during robotics competitions: the METRICS paradigm - Archive ouverte HAL
Communication Dans Un Congrès Année : 2020

AI evaluation campaigns during robotics competitions: the METRICS paradigm

Campagnes d'évaluation IA intégrées à des compétitions robotiques : le paradigme METRICS

Résumé

Competitions are a proven and cost-effective method to quickly develop new disruptive technologies for new markets. However, artificial intelligence and robotics competitions are usually organized independently, in silos. To get the most out of competitions, evaluation of robots should be modular (tasks are designed to assess independently different technological building blocks of the robotic systems) and open (evaluation tools and data must be publicly and freely available online). This not only meets the benchmarking needs of end users (mostly interested in performance measurements of the complete system), but also those of integrators and developers of intelligent components (camera, lidar, radar, actuators, etc.). With this objective in mind, the H2020 METRICS project (2020-2023) organizes competitions in four application areas (Healthcare, Infrastructure Inspection and Maintenance, Agri-Food, and Agile Production) relying on both physical testing facilities (field evaluation campaign) and virtual testing facilities (data-based evaluation campaign) to mobilize, in addition to the European robotics community, the artificial intelligence one. This article presents this approach, which aims to go beyond the METRICS project and pave the way for a new robotics and artificial intelligence competition paradigm.
Fichier principal
Vignette du fichier
METRICS_EPAI2020.pdf (396.99 Ko) Télécharger le fichier
Origine Fichiers produits par l'(les) auteur(s)

Dates et versions

hal-03009954 , version 1 (17-11-2020)

Identifiants

  • HAL Id : hal-03009954 , version 1

Citer

Guillaume Avrin, Virginie Barbosa, Agnes Delaborde. AI evaluation campaigns during robotics competitions: the METRICS paradigm. 1st international workshop on Evaluating Progress in Artificial Intelligence (EPAI 2020) in conjunction with ECAI 2020, Aug 2020, Santiago de Compostela, Spain. ⟨hal-03009954⟩

Collections

LNE
141 Consultations
72 Téléchargements

Partager

More