Dynamic Query Selection for Fast Visual Perceiver - Archive ouverte HAL Access content directly
Conference Papers Year : 2022

Dynamic Query Selection for Fast Visual Perceiver

Abstract

Transformers have been matching deep convolutional networks for vision architectures in recent works. Most work is focused on getting the best results on large-scale benchmarks, and scaling laws seem to be the most successful strategy: bigger models, more data, and longer training result in higher performance. However, the reduction of network complexity and inference time remains underexplored. The Perceiver model offers a solution to this problem: by first performing a Cross-attention with a fixed number Q of latent query tokens, the complexity of the L−layers Transformer network that follows is bounded by O(LQ 2). In this work, we explore how to make Perceivers even more efficient, by reducing the number of queries Q during inference while limiting the accuracy drop.
Fichier principal
Vignette du fichier
2205.10873.pdf (611.91 Ko) Télécharger le fichier
Origin : Files produced by the author(s)

Dates and versions

hal-03945064 , version 1 (18-01-2023)

Identifiers

Cite

Corentin Dancette, Matthieu Cord. Dynamic Query Selection for Fast Visual Perceiver. CVPR 2022 Transformer For Vision Workshop, Jun 2022, La nouvelle orléans, United States. ⟨hal-03945064⟩
4 View
1 Download

Altmetric

Share

Gmail Facebook Twitter LinkedIn More