Analyzing limits for in-context learning
Résumé
Our paper challenges claims from prior research that transformer-based models, when learning in context, implicitly implement standard learning algorithms. We present empirical evidence inconsistent with this view and provide a mathematical analysis demonstrating that transformers cannot achieve general predictive accuracy due to inherent architectural limitations.
Domaines
| Origine | Fichiers produits par l'(les) auteur(s) |
|---|---|
| Licence |