Communication Dans Un Congrès Année : 2025

Analyzing limits for in-context learning

Résumé

Our paper challenges claims from prior research that transformer-based models, when learning in context, implicitly implement standard learning algorithms. We present empirical evidence inconsistent with this view and provide a mathematical analysis demonstrating that transformers cannot achieve general predictive accuracy due to inherent architectural limitations.

Fichier principal
Vignette du fichier
icl_limits.pdf (906.58 Ko) Télécharger le fichier
Origine Fichiers produits par l'(les) auteur(s)
Licence

Dates et versions

hal-05374589 , version 1 (20-11-2025)

Licence

Identifiants

Citer

Omar Naim, Jérôme Bolte, Nicholas Asher. Analyzing limits for in-context learning. 39th Conference on Neural Information Processing Systems (NeurIPS 2025) Workshop: What Can(’t) Transformers Do?, Nov 2025, San Diego (California), United States. ⟨10.48550/arXiv.2502.03503⟩. ⟨hal-05374589⟩
2300 Consultations
254 Téléchargements

Altmetric

Partager

  • More