Benchmarking LLM-based Ontology Conceptualization: A Proposal - Archive ouverte HAL
Communication Dans Un Congrès Année : 2024

Benchmarking LLM-based Ontology Conceptualization: A Proposal

Youssra Rebboud
  • Fonction : Auteur
  • PersonId : 1157135
Pasquale Lisena
Lionel Tailhardat
  • Fonction : Auteur
  • PersonId : 1433872
Raphael Troncy

Résumé

This study presents a benchmark proposal designed to enhance knowledge engineering tasks through the use of large language models (LLMs). As LLMs become increasingly pivotal in knowledge extraction and modeling, it is crucial to evaluate and improve their performance. Building on prior work aiming at reverse generating competency questions (CQs) from existing ontologies, we introduce a benchmark focused on specific knowledge modeling tasks including ontology documentation, ontology generation, and query generation. In addition, we propose a baseline evaluation framework that applies various techniques, such as semantic comparison, ontology evaluation criteria, and structural comparison, using both existing ground truth datasets and newly proposed ontologies with corresponding CQs and documentation. This rigorous evaluation aims to provide a deeper understanding of LLM capabilities and contribute to their optimization in knowledge engineering applications.
Fichier principal
Vignette du fichier
ISWC_2024_Special_Session_on_LLM_for_KE (2).pdf (124.74 Ko) Télécharger le fichier
Origine Fichiers éditeurs autorisés sur une archive ouverte

Dates et versions

hal-04757816 , version 1 (29-10-2024)

Licence

Identifiants

  • HAL Id : hal-04757816 , version 1

Citer

Youssra Rebboud, Pasquale Lisena, Lionel Tailhardat, Raphael Troncy. Benchmarking LLM-based Ontology Conceptualization: A Proposal. ISWC 2024, 23rd International Semantic Web Conference, Nov 2024, Baltimore, United States. ⟨hal-04757816⟩

Collections

ANR
0 Consultations
0 Téléchargements

Partager

More