Challenges for operationalizing XAI in Critical Interactive Systems
Résumé
Large scale critical interactive systems (e.g. aircraft cockpits, satellite command and control applications…) aim to carry out complex missions. Users of such system usually perform predefined tasks for which they are trained and qualified. Nowadays, critical systems tend to embed an increasing number of on-board sensors, which collect large amounts of data to be interpreted and extrapolated, which tend to make more complex user tasks. Artificial Intelligence (AI) could be a powerful option to support the users in managing their tasks and handling this complexity. However, operationalizing AI in critical interactive systems requires proving that the AI behavior is consistent with user tasks, as well as transparent to the users and to the certification stakeholders. Explainable AI (XAI) is key as it could be a significant mean to satisfy this requirement. Nevertheless, at the same time, XAI will also have to comply with needs and common practices for the design and development of critical interactive systems. This position paper discusses the main challenges for operationalizing XAI in critical interactive systems.
Origine | Fichiers produits par l'(les) auteur(s) |
---|