Precise Minimax Regret for Logistic Regression - Archive ouverte HAL Access content directly
Conference Papers Year : 2022

Precise Minimax Regret for Logistic Regression


We study online logistic regression with binary labels and general feature values in which a learner sequentially tries to predict an outcome/ label based on data/ features received in rounds. Our goal is to evaluate precisely the (maximal) minimax regret which we analyze using a unique and novel combination of information-theoretic and analytic combinatoric tools such as Fourier transform, saddle point method, and Mellin transform in the multi-dimensional settings. To be more precise, the pointwise regret of an online algorithm is defined as the (excess) loss it incurs over a constant comparator (weight vector) that is used for prediction. It depends on the feature values, label sequence, and the learning algorithm. In the maximal minimax scenario we seek the best weights for the worst label sequence over all label distributions. For dimension d = o(T 1/3) we show that the maximal minimax regret grows as d 2 log(2T /π) + C d + O(d 3/2 / √ T) where T is the number of rounds of running a training algorithm and C d is explicitly computable constant that depends on dimension d and data. For features uniformly distributed on a d-dimensional sphere or ball we estimate precisely the constant C d showing that C d ∼ −(d/2) log(d/ √ 2π) leading to the minimax regret growing for large d as (d/2) log(T /d) − (d/2) log(√ 8π) + O(1). We also extend these results to non-binary labels. The precise maximal minimax regret presented here is the first result of this kind for any feature values and wide range of d. This provides a precise answer to the challenge posed in McMahan and Streeter (2010).
Fichier principal
Vignette du fichier
colt-regret.pdf (310.21 Ko) Télécharger le fichier
Origin : Files produced by the author(s)

Dates and versions

hal-03279834 , version 1 (06-07-2021)


  • HAL Id : hal-03279834 , version 1


Philippe Jacquet, Gil I. Shamir, Wojciech Szpankowski. Precise Minimax Regret for Logistic Regression. ISIT 2022, Jun 2022, Espoo, Finland. ⟨hal-03279834⟩
45 View
51 Download


Gmail Facebook Twitter LinkedIn More