A Conditional Randomization Test for Sparse Logistic Regression in High-Dimension - Archive ouverte HAL Access content directly
Conference Papers Year :

A Conditional Randomization Test for Sparse Logistic Regression in High-Dimension

(1, 2) , (3, 4, 5) , (6, 3, 7, 8)
1
2
3
4
5
6
7
8

Abstract

Identifying the relevant variables for a classification model with correct confidence levels is a central but difficult task in high-dimension. Despite the core role of sparse logistic regression in statistics and machine learning, it still lacks a good solution for accurate inference in the regime where the number of features p is as large as or larger than the number of samples n. Here we tackle this problem by improving the Conditional Randomization Test (CRT). The original CRT algorithm shows promise as a way to output p-values while making few assumptions on the distribution of the test statistics. As it comes with a prohibitive computational cost even in mildly high-dimensional problems, faster solutions based on distillation have been proposed. Yet, they rely on unrealistic hypotheses and result in low-power solutions. To improve this, we propose CRT-logit, an algorithm that combines a variable-distillation step and a decorrelation step that takes into account the geometry of $\ell_1$-penalized logistic regression problem. We provide a theoretical analysis of this procedure, and demonstrate its effectiveness on simulations, along with experiments on large-scale brain-imaging and genomics datasets.
Fichier principal
Vignette du fichier
large-crt-arxiv.pdf (2.49 Mo) Télécharger le fichier
Origin : Files produced by the author(s)

Dates and versions

hal-03680792 , version 1 (29-05-2022)

Identifiers

  • HAL Id : hal-03680792 , version 1

Cite

Binh T. Nguyen, Bertrand Thirion, Sylvain Arlot. A Conditional Randomization Test for Sparse Logistic Regression in High-Dimension. NeurIPS 2022, Nov 2022, New Orleans, United States. ⟨hal-03680792⟩
159 View
9 Download

Share

Gmail Facebook Twitter LinkedIn More