Communication Dans Un Congrès Année : 2026

SEF-CLGC at SemEval-2026 Task 11: Logical Notation Impact on Language Model Performance

Résumé

This paper revisits our pipeline called Syllogistic Evaluation Framework-Common Logic Grammar Construction (SEF-CLGC). We combine formal logical notations with Small Language Models (SLMs) to evaluate reasoning performance on the SemEval-2026 Task 11 Subtask 1: Disentangling Content and Formal Reasoning in Large Language Models. Our experiments show that by relying solely on SLMs, trained on a combination of natural and symbolic languages, our best model achieves a content score of 27.80% on the task while significantly lowering the content bias in reasoning.

Fichier principal
Vignette du fichier
SEF_CLGC_at_SemEval_2026_Task_11__Logical_Notations_Impact_on_Language_Model_Performance_CameraReady.pdf (434.97 Ko) Télécharger le fichier
Origine Fichiers produits par l'(les) auteur(s)
Licence

Dates et versions

hal-05606250 , version 1 (29-04-2026)

Licence

Identifiants

  • HAL Id : hal-05606250 , version 1

Citer

Hanna Abi Akl, Fabien Gandon, Catherine Faron, Pierre Monnin. SEF-CLGC at SemEval-2026 Task 11: Logical Notation Impact on Language Model Performance. SemEval-2026 - The 20th International Workshop on Semantic Evaluation, Jul 2026, San Diego (California), United States. ⟨hal-05606250⟩
16 Consultations
1 Téléchargements

Partager

  • More