AI evaluation campaigns during robotics competitions: the METRICS paradigm
Campagnes d'évaluation IA intégrées à des compétitions robotiques : le paradigme METRICS
Résumé
Competitions are a proven and cost-effective method to quickly develop new disruptive technologies for new markets. However, artificial intelligence and robotics competitions are usually organized independently, in silos. To get the most out of competitions, evaluation of robots should be modular (tasks are designed to assess independently different technological building blocks of the robotic systems) and open (evaluation tools and data must be publicly and freely available online). This not only meets the benchmarking needs of end users (mostly interested in performance measurements of the complete system), but also those of integrators and developers of intelligent components (camera, lidar, radar, actuators, etc.).
With this objective in mind, the H2020 METRICS project (2020-2023) organizes competitions in four application areas (Healthcare, Infrastructure Inspection and Maintenance, Agri-Food, and Agile Production) relying on both physical testing facilities (field evaluation campaign) and virtual testing facilities (data-based evaluation campaign) to mobilize, in addition to the European robotics community, the artificial intelligence one.
This article presents this approach, which aims to go beyond the METRICS project and pave the way for a new robotics and artificial intelligence competition paradigm.
Domaines
Intelligence artificielle [cs.AI]Origine | Fichiers produits par l'(les) auteur(s) |
---|