Online A-optimal design and active linear regression - Archive ouverte HAL Accéder directement au contenu
Communication Dans Un Congrès Année : 2021

Online A-optimal design and active linear regression

Xavier Fontaine
  • Fonction : Auteur
  • PersonId : 1063788
Pierre Perrault
  • Fonction : Auteur
Michal Valko

Résumé

We consider in this paper the problem of optimal experiment design where a decision maker can choose which points to sample to obtain an estimateβ of the hidden parameter β of an underlying linear model. The key challenge of this work lies in the heteroscedasticity assumption that we make, meaning that each covariate has a different and unknown variance. The goal of the decision maker is then to figure out on the fly the optimal way to allocate the total budget of T samples between covariates, as sampling several times a specific one will reduce the variance of the estimated model around it (but at the cost of a possible higher variance elsewhere). By trying to minimize the 2-loss E[ β − β 2 ] the decision maker is actually minimizing the trace of the covariance matrix of the problem, which corresponds then to online A-optimal design. Combining techniques from bandit and convex optimization we propose a new active sampling algorithm and we compare it with existing ones. We provide theoretical guarantees of this algorithm in different settings, including a O(T −2) regret bound in the case where the covariates form a basis of the feature space, generalizing and improving existing results. Numerical experiments validate our theoretical findings.
Fichier principal
Vignette du fichier
fontaine2021online.pdf (2.42 Mo) Télécharger le fichier
Origine : Fichiers produits par l'(les) auteur(s)

Dates et versions

hal-03289318 , version 1 (16-07-2021)

Identifiants

  • HAL Id : hal-03289318 , version 1

Citer

Xavier Fontaine, Pierre Perrault, Michal Valko, Vianney Perchet. Online A-optimal design and active linear regression. International Conference on Machine Learning, Jul 2021, Vienna / Virtual, Austria. ⟨hal-03289318⟩
85 Consultations
151 Téléchargements

Partager

Gmail Facebook X LinkedIn More