Communication Dans Un Congrès Année : 2025

Towards Multi-Document Question Answering in Scientific Literature: Pipeline, Dataset, and Evaluation

Résumé

Question-Answering (QA) systems are vital for rapidly accessing and comprehending information in academic literature.However, some academic questions require synthesizing information across multiple documents. While several prior resources consider multi-document QA, they often do not strictly enforce cross-document synthesis or exploit the explicit inter-paper structure that links sources.To address this, we introduce a pipeline methodology for constructing a Multi-Document Academic QA (MDA-QA) dataset. By both detecting communities based on citation networks and leveraging Large Language Models (LLMs), we were able to form thematically coherent communities and generate QA pairs related to multi-document content automatically.We further develop an automated filtering mechanism to ensure multi-document dependence.Our resulting dataset consists of 6,804 QA pairs and serves as a benchmark for evaluating multi-document retrieval and QA systems.Our experimental results highlight that standard lexical and embedding-based retrieval methods struggle to locate all relevant documents, indicating a persistent gap in multi-document reasoning. We release our dataset and source code for the community.

Fichier principal
Vignette du fichier
2025.findings-emnlp.576v2.pdf (1.17 Mo) Télécharger le fichier
Origine Fichiers produits par l'(les) auteur(s)
Licence

Dates et versions

hal-05579336 , version 1 (03-04-2026)

Licence

Identifiants

Citer

Hui Huang, Julien Velcin, Yacine Kessaci. Towards Multi-Document Question Answering in Scientific Literature: Pipeline, Dataset, and Evaluation. Findings of the Association for Computational Linguistics: EMNLP 2025, Nov 2025, Suzhou, China. pp.10867-10881, ⟨10.18653/v1/2025.findings-emnlp.576⟩. ⟨hal-05579336⟩
36 Consultations
18 Téléchargements

Altmetric

Partager

  • More