Ultra-Fast Machine Learning Inference through C Code Generation for Tangled Program Graphs
Résumé
Tangled Program Graph (TPG) is a Reinforcement Learning (RL) technique based on genetic programming concepts. On state-of-the-art learning environments, TPGs have been shown to offer comparable competence with Deep Neural Networks (DNNs), for a fraction of their computational and storage cost. The contribution of this paper focuses on accelerating the inference of pre-trained TPGs, through the generation of standalone C code. While the training process of TPGs, based on genetic evolution principles, requires the use of flexible data structures supporting random mutations, this flexibility is no longer needed when focusing on the inference process. Evaluation of the proposed approach on four computing platforms, including embedded CPUs, produces an acceleration of the TPG inference by a factor 50 compared to state-ofthe-art implementations. The inference performance obtained within a complex RL environment range between hundreds of nano-seconds to micro-seconds, making this approach highly competitive for edge Artificial Intelligence (AI).
Origine | Fichiers produits par l'(les) auteur(s) |
---|