Communication Dans Un Congrès Année : 2025

Beyond Similarity Scoring: Detecting Entailment and Contradiction in Multilingual and Multimodal Contexts

Résumé

Natural Language Inference (NLI) determines whether a hypothesis entails, contradicts, or is neutral with respect to a premise. While text-based NLI is well-studied, its multimodal and multilingual extension remains underexplored. This paper introduces a multilingual, multimodal NLI framework classifying entailment, contradiction, and neutrality across text-text, text-speech, speech-text, and speech-speech pairs in same-and cross-lingual settings. A key motivation is improving translation assessment, where similarity-based approaches may miss contradictions. The framework complements evaluation methods and helps identify inconsistencies by detecting entailment and contradiction alongside semantic similarity. It also extends text-based datasets with speech-text and speech-speech pairs for multilingual multimodal inference. 1 Experiments show the model outperforms BLASER in distinguishing entailment from non-entailment, achieving F1 gains of 0.19 in speech-speech and 0.13 in speech-text.

Fichier principal
Vignette du fichier
Beyond_Similarity_Scoring__Detecting_Entailment_and_Contradiction_in_Multilingual_and_Multimodal_Contexts.pdf (816.01 Ko) Télécharger le fichier
Origine Fichiers produits par l'(les) auteur(s)
Licence

Dates et versions

hal-05493381 , version 1 (04-02-2026)

Licence

Identifiants

Citer

Othman Istaiteh, Salima Mdhaffar, Yannick Estève. Beyond Similarity Scoring: Detecting Entailment and Contradiction in Multilingual and Multimodal Contexts. Interspeech 2025, Aug 2025, Rotterdam (NL), France. pp.286-290, ⟨10.21437/Interspeech.2025-1825⟩. ⟨hal-05493381⟩

Collections

38 Consultations
40 Téléchargements

Altmetric

Partager

  • More