Affine Invariant Covariance Estimation for Heavy-Tailed Distributions - Archive ouverte HAL Accéder directement au contenu
Communication Dans Un Congrès Année : 2019

Affine Invariant Covariance Estimation for Heavy-Tailed Distributions

Résumé

In this work we provide an estimator for the covariance matrix of a heavy-tailed multivariate distribution We prove that the proposed estimator $\widehat{\mathbf{S}}$ admits an \textit{affine-invariant} bound of the form \[ (1-\varepsilon) \mathbf{S} \preccurlyeq \widehat{\mathbf{S}} \preccurlyeq (1+\varepsilon) \mathbf{S} \] in high probability, where $\mathbf{S}$ is the unknown covariance matrix, and $\preccurlyeq$ is the positive semidefinite order on symmetric matrices. The result only requires the existence of fourth-order moments, and allows for $\varepsilon = O(\sqrt{\kappa^4 d\log(d/\delta)/n})$ where $\kappa^4$ is a measure of kurtosis of the distribution, $d$ is the dimensionality of the space, $n$ is the sample size, and $1-\delta$ is the desired confidence level. More generally, we can allow for regularization with level $\lambda$, then $d$ gets replaced with the degrees of freedom number. Denoting $\text{cond}(\mathbf{S})$ the condition number of $\mathbf{S}$, the computational cost of the novel estimator is $O(d^2 n + d^3\log(\text{cond}(\mathbf{S})))$, which is comparable to the cost of the sample covariance estimator in the statistically interesing regime $n \ge d$. We consider applications of our estimator to eigenvalue estimation with relative error, and to ridge regression with heavy-tailed random design.
Fichier principal
Vignette du fichier
heavy-covariance-eprint.pdf (457.18 Ko) Télécharger le fichier
Origine : Fichiers produits par l'(les) auteur(s)
Loading...

Dates et versions

hal-02011464 , version 1 (08-02-2019)
hal-02011464 , version 2 (23-09-2019)

Licence

Domaine public

Identifiants

Citer

Dmitrii M. Ostrovskii, Alessandro Rudi. Affine Invariant Covariance Estimation for Heavy-Tailed Distributions. COLT 2019 - 32nd Annual Conference on Learning Theory, Jun 2019, Phoenix, United States. ⟨hal-02011464v2⟩
383 Consultations
153 Téléchargements

Altmetric

Partager

Gmail Facebook X LinkedIn More