Communication Dans Un Congrès Année : 2024

Evaluating the Effectiveness of Large Language Models in Establishing Conversational Grounding

Résumé

Conversational grounding, vital for building effective dialogue between people and between people and dialogue systems, involves ensuring a mutual understanding of shared information. Despite its importance, there has been limited research on this aspect of conversation in recent years, especially after the advent of Large Language Models (LLMs). Previous studies have highlighted the shortcomings of some pre-trained language models in conversational grounding. However, most testing for conversational grounding capabilities involves human evaluations that are costly and time-consuming. This has led to a lack of testing across multiple models of varying sizes, a critical need given the rapid rate of new model releases. This gap in research becomes more significant considering recent advances in language models, which have led to new emergent capabilities. In this paper, we evaluate the performance of LLMs in various aspects of conversational grounding and analyze why some models perform better than others. We demonstrate a direct correlation between the size of the pre-training dataset, size of the model and conversational grounding abilities, suggesting that they have independently acquired some pragmatic capabilities from larger pre-training datasets. Finally, we propose ways to enhance the capabilities of the models that lag in our tests.

Fichier principal
Vignette du fichier
Biswesh___EMNLP_2024 (3).pdf (324.36 Ko) Télécharger le fichier
Origine Fichiers produits par l'(les) auteur(s)
Licence

Dates et versions

hal-04881738 , version 1 (13-01-2025)

Licence

Identifiants

Citer

Biswesh Mohapatra, Manav Nitin Kapadnis, Laurent Romary, Justine Cassell. Evaluating the Effectiveness of Large Language Models in Establishing Conversational Grounding. EMNLP 2024 - Conference on Empirical Methods in Natural Language Processing, Association of Computational Linguistics, Nov 2024, Miami, United States. pp.9767-9781, ⟨10.18653/v1/2024.emnlp-main.545⟩. ⟨hal-04881738⟩
354 Consultations
331 Téléchargements

Altmetric

Partager

  • More