Investigating the Effect of Relative Positional Embeddings on AMR-to-Text Generation with Structural Adapters - Archive ouverte HAL Accéder directement au contenu
Communication Dans Un Congrès Année : 2023

Investigating the Effect of Relative Positional Embeddings on AMR-to-Text Generation with Structural Adapters

Résumé

Text generation from Abstract Meaning Representation (AMR) has substantially benefited from the popularized Pretrained Language Models (PLMs). Myriad approaches have linearized the input graph as a sequence of tokens to fit the PLM tokenization requirements. Nevertheless, this transformation jeopardizes the structural integrity of the graph and is therefore detrimental to its resulting representation. To overcome this issue, Ribeiro et al. (2021b) have recently proposed StructAdapt, a structure-aware adapter which injects the input graph connectivity within PLMs using Graph Neural Networks (GNNs). In this paper, we investigate the influence of Relative Position Embeddings (RPE) on AMR-to-Text, and, in parallel, we examine the robustness of Struc-tAdapt. Through ablation studies, graph attack and link prediction, we reveal that RPE might be partially encoding input graphs. We suggest further research regarding the role of RPE will provide valuable insights for Graph-to-Text generation.
2023.eacl-main.51.pdf (477.17 Ko) Télécharger le fichier

Dates et versions

hal-04151438 , version 1 (04-07-2023)

Identifiants

  • HAL Id : hal-04151438 , version 1

Citer

Sebastien Montella, Alexis Nasr, Johannes Heinecke, Frederic Bechet, Lina M Rojas-Barahona. Investigating the Effect of Relative Positional Embeddings on AMR-to-Text Generation with Structural Adapters. 17th Conference of the European Chapter of the Association for Computational Linguistics, EACL, May 2023, Dubrovnik, Croatia. pp.727-736. ⟨hal-04151438⟩
4 Consultations
3 Téléchargements

Partager

Gmail Facebook X LinkedIn More