Generation of Training Examples for Tabular Natural Language Inference - Archive ouverte HAL Access content directly
Journal Articles Proceedings of the ACM on Management of Data Year : 2023

Generation of Training Examples for Tabular Natural Language Inference

Abstract

Tabular data is becoming increasingly important in Natural Language Processing (NLP) tasks, such as Tabular Natural Language Inference (TNLI). Given a table and a hypothesis expressed in NL text, the goal is to assess if the former structured data supports or refutes the latter. In this work, we focus on the role played by the annotated data in training the inference model. We introduce a system, Tenet, for the automatic augmentation and generation of training examples for TNLI. Given the tables, existing approaches are either based on human annotators, and thus expensive, or on methods that produce simple examples that lack data variety and complex reasoning. Instead, our approach is built around the intuition that SQL queries are the right tool to achieve variety in the generated examples, both in terms of data variety and reasoning complexity. The first is achieved by evidence-queries that identify cell values over tables according to different data patterns. Once the data for the example is identified, semantic-queries describe the different ways such data can be identified with standard SQL clauses. These rich descriptions are then verbalized as text to create the annotated examples for the TNLI task. The same approach is also extended to create counterfactual examples, i.e., examples where the hypothesis is false, with a method based on injecting errors in the original (clean) table. For all steps, we introduce generic generation algorithms that take as input only the tables. For our experimental study, we use three datasets from the TNLI literature and two crafted by us on more complex tables. Tenet generates human-like examples, which lead to the effective training of several inference models with results comparable to those obtained by training the same models with manually-written examples.
Fichier principal
Vignette du fichier
publi-7476.pdf (893.53 Ko) Télécharger le fichier
Origin Publisher files allowed on an open archive

Dates and versions

hal-04249392 , version 1 (19-10-2023)

Identifiers

Cite

Jean-Flavien Bussotti, Enzo Veltri, Donatello Santoro, Paolo Papotti. Generation of Training Examples for Tabular Natural Language Inference. Proceedings of the ACM on Management of Data, In press, 1 (4), ⟨10.1145/3626730⟩. ⟨hal-04249392⟩

Collections

EURECOM ANR
112 View
125 Download

Altmetric

Share

Gmail Mastodon Facebook X LinkedIn More