Towards Fact-check Summarization Leveraging on Argumentation Elements Tied to Entity Graphs - Archive ouverte HAL
Communication Dans Un Congrès Année : 2024

Towards Fact-check Summarization Leveraging on Argumentation Elements Tied to Entity Graphs

Résumé

Fact-check consumers can have different preferences regarding the amount of text being used for explaining the claim veracity verdict. Dynamically adapting the size of a fact-check report is thus an important functionality for systems designed to convey claim verification explainability. Recent works have experimented with applying transformers-based or LLM-based text summarization methods in a zero-shot or few-shot manner, making use of some existing texts available in the summary parts of fact-check reports (e.g., called "justification'' in PolitiFact). However, for complex fact-checks, the purely sub-symbolic summarizers tend to either omit some elements of the fact-checker's argumentation chains or include contextual statements that may not be essential at the given level of granularity. In this paper, we propose a new method for enhancing fact-check summarization with the aim of injecting elements of structured fact-checker argumentation. This argumentation is, in turn, not only captured at the discourse level but tied to an entity graph representing the fact-check, for which we employ the PURO diagrammatic language. We have empirically performed a manual analysis of fact-check reports from two fact-checker websites, yielding (1) textual snippets containing the argumentation essence of the fact-check report and (2) categorized argumentation elements tied to entity graphs. These snippets are then fed to a state-of-the-art hybrid summarizer which has previously produced accurate fact-check summaries, as an additional input. We observe mild improvements on various ROUGE metrics, even if the validity of the results is limited given the small size of the dataset. We also compare the human-provided argumentation element categories with those returned, for the given fact-check ground truth summary, using a pre-trained language model upon both basic and augmented prompting. This yields a moderate accuracy as the model often fails to comply with the explicit given instructions.
Fichier non déposé

Dates et versions

hal-04576180 , version 1 (15-05-2024)

Identifiants

Citer

Kateřina Haniková, David Chudán, Vojtěch Svátek, Peter Vajdečka, Raphaël Troncy, et al.. Towards Fact-check Summarization Leveraging on Argumentation Elements Tied to Entity Graphs. WWW 2024, ACM Web Conference 2024, May 2024, Singapore (SG), Singapore. pp.1473-1481, ⟨10.1145/3589335.3651914⟩. ⟨hal-04576180⟩

Collections

EURECOM ANR
21 Consultations
0 Téléchargements

Altmetric

Partager

More