Evaluate on-the-job learning dialogue systems and a case study for natural language understanding - Archive ouverte HAL
Communication Dans Un Congrès Année : 2020

Evaluate on-the-job learning dialogue systems and a case study for natural language understanding

Évaluer les systèmes de dialogue apprenant sur le terrain et un cas d'étude pour la compréhension du langage naturel

Résumé

On-the-job learning consists in continuously learning while being used in production, in an open environment, meaning that the system has to deal on its own with situations and elements never seen before. The kind of systems that seem to be especially adapted to on-the-job learning are dialogue systems, since they can take advantage of their interactions with users to collect feedback to adapt and improve their components over time. Some dialogue systems performing on-the-job learning have been built and evaluated but no general methodology has yet been defined. Thus in this paper, we propose a first general methodology for evaluating on-the-job learning dialogue systems. We also describe a task-oriented dialogue system which improves on-the-job its natural language component through its user interactions. We finally evaluate our system with the described methodology.
Fichier principal
Vignette du fichier
NeurIPS_2020_HLDS_on_the_job_NLU_v4_camera_ready.pdf (440.48 Ko) Télécharger le fichier
Origine Fichiers produits par l'(les) auteur(s)

Dates et versions

hal-03301611 , version 1 (27-07-2021)

Identifiants

Citer

Mathilde Veron, Sophie Rosset, Olivier Galibert, Guillaume Bernard. Evaluate on-the-job learning dialogue systems and a case study for natural language understanding. Workshop NeurIPS 2020 Human in the Loop Dialogue Systems, Dec 2020, Virtual only, United States. ⟨hal-03301611⟩
110 Consultations
44 Téléchargements

Altmetric

Partager

More