Communication Dans Un Congrès Année : 2025

Easing Optimization Paths: a Circuit Perspective

Résumé

Gradient descent is the method of choice for training large artificial intelligence systems. As these systems become larger, a better understanding of the mechanisms behind gradient training would allow us to alleviate compute costs and help steer these systems away from harmful behaviors. To that end, we suggest utilizing the circuit perspective brought forward by mechanistic interpretability. After laying out our intuition, we illustrate how it enables us to design a curriculum for efficient learning in a controlled setting. The code is available at \url{https://github.com/facebookresearch/pal}.

Fichier principal
Vignette du fichier
2501.02362v1.pdf (718.23 Ko) Télécharger le fichier
Origine Fichiers produits par l'(les) auteur(s)
Licence

Dates et versions

hal-05069172 , version 1 (15-05-2025)

Licence

Identifiants

Citer

Ambroise Odonnat, Wassim Bouaziz, Vivien Cabannes. Easing Optimization Paths: a Circuit Perspective. ICASSP 2025 - IEEE International Conference on Acoustics, Speech and Signal Processing, Apr 2025, Hyderabab, India. pp.1-5, ⟨10.1109/icassp49660.2025.10888894⟩. ⟨hal-05069172⟩
95 Consultations
170 Téléchargements

Altmetric

Partager

  • More