Relational Data Embeddings for Feature Enrichment with Background Information - Archive ouverte HAL Access content directly
Journal Articles Machine Learning Year : 2023

Relational Data Embeddings for Feature Enrichment with Background Information


For many machine-learning tasks, augmenting the data table at hand with features built from external sources is key to improving performance. For instance, estimating housing prices benefits from background information on the location, such as the population density or the average income. However, this information must often be assembled across many tables, requiring time and expertise from the data scientist. Instead, we propose to replace human-crafted features by vectorial representations of entities (e.g. cities) that capture the corresponding information. We represent the relational data on the entities as a graph and adapt graph-embedding methods to create feature vectors for each entity. We show that two technical ingredients are crucial: modeling well the different relationships between entities, and capturing numerical attributes. We adapt knowledge graph embedding methods that were primarily designed for graph completion. Yet, they model only discrete entities, while creating good feature vectors from relational data also requires capturing numerical attributes. For this, we introduce KEN: Knowledge Embedding with Numbers. We thoroughly evaluate approaches to enrich features with background information on 7 prediction tasks. We show that a good embedding model coupled with KEN can perform better than manually handcrafted features, while requiring much less human effort. It is also competitive with combinatorial feature engineering methods, but much more scalable. Our approach can be applied to huge databases, creating general-purpose feature vectors reusable in various downstream tasks.
Fichier principal
Vignette du fichier
main.pdf (1.53 Mo) Télécharger le fichier
Origin : Files produced by the author(s)

Dates and versions

hal-03848124 , version 1 (10-11-2022)



Alexis Cvetkov-Iliev, Alexandre Allauzen, Gaël Varoquaux. Relational Data Embeddings for Feature Enrichment with Background Information. Machine Learning, 2023, 112 (2), pp.687-720. ⟨10.1007/s10994-022-06277-7⟩. ⟨hal-03848124⟩
529 View
666 Download



Gmail Facebook X LinkedIn More