User Scored Evaluation of Non-Unique Explanations for Relational Graph Convolutional Network Link Prediction on Knowledge Graphs - Archive ouverte HAL Access content directly
Conference Papers Year : 2021

User Scored Evaluation of Non-Unique Explanations for Relational Graph Convolutional Network Link Prediction on Knowledge Graphs

Abstract

Relational Graph Convolutional Networks (RGCNs) are commonly used on Knowledge Graphs (KGs) to perform black box link prediction. Several algorithms, or explanation methods, have been proposed to explain their predictions. Evaluating performance of explanation methods for link prediction is difficult without ground truth explanations. Furthermore, there can be multiple explanations for a given prediction in a KG. No dataset exists where observations have multiple ground truth explanations to compare against. Additionally, no standard scoring metrics exist to compare predicted explanations against multiple ground truth explanations. In this paper, we introduce a method, including a dataset (FrenchRoyalty-200k), to benchmark explanation methods on the task of link prediction on KGs, when there are multiple explanations to consider. We conduct a user experiment, where users score each possible ground truth explanation based on their understanding of the explanation. We propose the use of several scoring metrics, using relevance weights derived from user scores for each predicted explanation. Lastly, we benchmark this dataset on state-of-the-art explanation methods for link prediction using the proposed scoring metrics.
Fichier principal
Vignette du fichier
K_CAP_2021.pdf (572.43 Ko) Télécharger le fichier
Origin : Files produced by the author(s)

Dates and versions

hal-03402766 , version 1 (26-10-2021)

Identifiers

  • HAL Id : hal-03402766 , version 1

Cite

Nicholas Halliwell, Fabien Gandon, Freddy Lecue. User Scored Evaluation of Non-Unique Explanations for Relational Graph Convolutional Network Link Prediction on Knowledge Graphs. International Conference on Knowledge Capture, Dec 2021, Virtual Event, United States. ⟨hal-03402766⟩
208 View
226 Download

Share

Gmail Facebook X LinkedIn More