SNEkhorn: Dimension Reduction with Symmetric Entropic Affinities - UFR ISTIC Informatique et électronique Accéder directement au contenu
Pré-Publication, Document De Travail Année : 2023

SNEkhorn: Dimension Reduction with Symmetric Entropic Affinities

Résumé

Many approaches in machine learning rely on a weighted graph to encode the similarities between samples in a dataset. Entropic affinities (EAs), which are notably used in the popular Dimensionality Reduction (DR) algorithm t-SNE, are particular instances of such graphs. To ensure robustness to heterogeneous sampling densities, EAs assign a kernel bandwidth parameter to every sample in such a way that the entropy of each row in the affinity matrix is kept constant at a specific value, whose exponential is known as perplexity. EAs are inherently asymmetric and row-wise stochastic, but they are used in DR approaches after undergoing heuristic symmetrization methods that violate both the row-wise constant entropy and stochasticity properties. In this work, we uncover a novel characterization of EA as an optimal transport problem, allowing a natural symmetrization that can be computed efficiently using dual ascent. The corresponding novel affinity matrix derives advantages from symmetric doubly stochastic normalization in terms of clustering performance, while also effectively controlling the entropy of each row thus making it particularly robust to varying noise levels. Following, we present a new DR algorithm, SNEkhorn, that leverages this new affinity matrix. We show its clear superiority to state-of-the-art approaches with several indicators on both synthetic and real-world datasets.
Fichier principal
Vignette du fichier
paper.pdf (1.29 Mo) Télécharger le fichier
Origine : Fichiers produits par l'(les) auteur(s)

Dates et versions

hal-04103326 , version 1 (23-05-2023)
hal-04103326 , version 2 (27-10-2023)

Identifiants

  • HAL Id : hal-04103326 , version 1

Citer

Hugues Van Assel, Titouan Vayer, Rémi Flamary, Nicolas Courty. SNEkhorn: Dimension Reduction with Symmetric Entropic Affinities. 2023. ⟨hal-04103326v1⟩
176 Consultations
70 Téléchargements

Partager

Gmail Facebook X LinkedIn More