SNEkhorn: Dimension Reduction with Symmetric Entropic Affinities - Archive ouverte HAL Access content directly
Conference Papers Year : 2023

SNEkhorn: Dimension Reduction with Symmetric Entropic Affinities

Abstract

Many approaches in machine learning rely on a weighted graph to encode the similarities between samples in a dataset. Entropic affinities (EAs), which are notably used in the popular Dimensionality Reduction (DR) algorithm t-SNE, are particular instances of such graphs. To ensure robustness to heterogeneous sampling densities, EAs assign a kernel bandwidth parameter to every sample in such a way that the entropy of each row in the affinity matrix is kept constant at a specific value, whose exponential is known as perplexity. EAs are inherently asymmetric and row-wise stochastic, but they are used in DR approaches after undergoing heuristic symmetrization methods that violate both the row-wise constant entropy and stochasticity properties. In this work, we uncover a novel characterization of EA as an optimal transport problem, allowing a natural symmetrization that can be computed efficiently using dual ascent. The corresponding novel affinity matrix derives advantages from symmetric doubly stochastic normalization in terms of clustering performance, while also effectively controlling the entropy of each row thus making it particularly robust to varying noise levels. Following, we present a new DR algorithm, SNEkhorn, that leverages this new affinity matrix. We show its clear superiority to state-of-the-art approaches with several indicators on both synthetic and real-world datasets.
Fichier principal
Vignette du fichier
final_snekhorn.pdf (1.83 Mo) Télécharger le fichier
Origin : Files produced by the author(s)

Dates and versions

hal-04103326 , version 1 (23-05-2023)
hal-04103326 , version 2 (27-10-2023)

Licence

Attribution

Identifiers

  • HAL Id : hal-04103326 , version 2

Cite

Hugues Van Assel, Titouan Vayer, Rémi Flamary, Nicolas Courty. SNEkhorn: Dimension Reduction with Symmetric Entropic Affinities. Thirty-seventh Annual Conference on Neural Information Processing Systems (NeurIPS), Dec 2023, New Orleans, United States. ⟨hal-04103326v2⟩
178 View
73 Download

Share

Gmail Facebook X LinkedIn More