Article Dans Une Revue Information Processing Letters Année : 2025

An Information Theoretic Proof of the Chernoff-Hoeffding Inequality

Résumé

The Chernoff bound is a well-known upper bound on the tail of binomial distributions of parameter 1/2 involving the binary entropy function. Hoeffding's inequality (or the Chernoff-Hoeffding inequality) is a generalization for binomial distributions of parameter 1 -1/q, involving the q-ary entropy function (with q ≥ 2), which can be written in terms of the Kullback-Leibler divergence and is related to the bound in Fano's inequality. We give an information theoretic proof of that bound, and sketch some applications to channel and source coding. We also derive a refined bound which is always sharper.

Fichier principal
Vignette du fichier
Rioul.pdf (324.04 Ko) Télécharger le fichier
Origine Fichiers produits par l'(les) auteur(s)
Licence

Dates et versions

hal-05047248 , version 1 (08-05-2025)

Licence

Identifiants

Citer

Olivier Rioul, Patrick Solé. An Information Theoretic Proof of the Chernoff-Hoeffding Inequality. Information Processing Letters, 2025, 190, pp.106582. ⟨10.1016/j.ipl.2025.106582⟩. ⟨hal-05047248⟩
4830 Consultations
212 Téléchargements

Altmetric

Partager

  • More