Integrating Semantic Representations in a Cross-Modal Approach to Fact-Checking
Résumé
We propose a cross-modal approach with deep fusion of a language model and graph structures based on Abstract Meaning Representa- tions (AMRs) enriched with Wikidata to address the fact-checking problem. We collect and make available a large dataset of fact- checked claim sentences, and systematically compare a transformer- based model with Graph Neural Networks (GNNs) based on AMR graphs and extended with external information. Furthermore, we evaluate the integration of language models and GNNs for the fact verification task. While GNN models on AMR-based graphs alone yield lower scores than transformer based language mod- els on their own, the combined cross-modal approach—leveraging a multilayer and deep interaction between textual and structural information—demonstrates the best performance. Finally, we eval- uate the generalization capability of this cross-modal approach integrating AMR-based graph structures on out-of-domain English and German claims.