Communication Dans Un Congrès Année : 2025

The Engineering of AI Evaluation and Scoring: Overview and Insights

Henri Sohier
Jean-Philippe Faure
  • Fonction : Auteur
  • PersonId : 1515894
Sebastian Hallensleben
  • Fonction : Auteur
  • PersonId : 1515895
Philippe Streiff
  • Fonction : Auteur
  • PersonId : 1515896
Stephen Creff
Emilie Sirvent-Hien
  • Fonction : Auteur
  • PersonId : 1128976
Juliette Mattioli
Serkan Odabas
  • Fonction : Auteur
  • PersonId : 1515897
Romane Vernhes
  • Fonction : Auteur
  • PersonId : 1515898

Résumé

The rapid evolution of Artificial Intelligence (AI) requires a rethinking of practices and better anticipation of impacts, as highlighted by several new regulations. Different communities, including engineers, auditors, business leaders, standardization experts, and ethicists, are developing AI evaluation frameworks with scoring systems to progressively operationalize good practices for AI. The development of AI evaluation frameworks should be closely tied to disciplines such as systems and software engineering, requirements engineering, quality management, risk analysis, verification and validation, and decision support. This paper formalizes the development of AI evaluation frameworks as six non-linear activities to better leverage these disciplines. It is intended to be of particular interest to communities engaged in AI evaluation but not familiar with systems engineering. It draws on the authors' experience in defining end-to-end methodologies for AI trustworthiness and designing evaluation frameworks. The activities presented are: (1) Managing the evaluation framework as a product in its own right; (2) Managing the evaluation framework as a generic specification for different objects; (3) Formalizing progressive good practices, possibly in the form of requirement levels; (4) Clustering these requirements into understandable dimensions; (5) Defining a global scoring scheme; and (6) Tailoring the framework to account for industry-specific considerations and risks. This formalization aims to enhance the effectiveness of AI evaluation frameworks.

Fichier principal
Vignette du fichier
Engineering_AI_Evaluation_Scoring.pdf (389.89 Ko) Télécharger le fichier
Origine Fichiers produits par l'(les) auteur(s)
Licence

Dates et versions

hal-04977577 , version 1 (11-04-2025)

Licence

Identifiants

  • HAL Id : hal-04977577 , version 1

Citer

Henri Sohier, Jean-Philippe Faure, Sebastian Hallensleben, Philippe Streiff, Stephen Creff, et al.. The Engineering of AI Evaluation and Scoring: Overview and Insights. 2025 IEEE International Systems Conference (SysCon), Apr 2025, Montreal, Canada. ⟨hal-04977577⟩
1586 Consultations
552 Téléchargements

Partager

  • More