An Evaluation Bench for the Exploration of Machine Learning Deployment Solutions on Embedded Platforms
Résumé
Finding the most efficient deployment of a Machine Learning (ML) model requires setting up multiple combinations of ML tools and hardware targets, running series of experiments, and evaluating relevant parameters (latency, memory usage, etc.).
All these operations are complex, sometimes tedious, and always time consuming. Therefore, in order to facilitate this Design Space Exploration process, we propose an evaluation bench that (i) integrates the necessary software and hardware resources (tools, boards) to deploy a variety of ML models, and (ii) provides a uniform and abstract API to exercise and evaluate multiple deployment solutions. This paper defines more precisely the end- users needs, describes the architecture of the bench and illustrates its application on a use case.