A Deep Neural Networks ensemble workflow from hyperparameter search to inference leveraging GPU clusters - Archive ouverte HAL Accéder directement au contenu
Communication Dans Un Congrès Année : 2022

A Deep Neural Networks ensemble workflow from hyperparameter search to inference leveraging GPU clusters

Pierrick Pochelu
  • Fonction : Auteur
  • PersonId : 1120905
Bruno Conche
  • Fonction : Auteur

Résumé

Automated Machine Learning with ensembling (or AutoML with ensembling) seeks to automatically build ensembles of Deep Neural Networks (DNNs) to achieve qualitative predictions. Ensemble of DNNs are well known to avoid over-fitting but they are memory and time consuming approaches. Therefore, an ideal AutoML would produce in one single run time different ensembles regarding accuracy and inference speed. While previous works on AutoML focus to search for the best model to maximize its generalization ability, we rather propose a new AutoML to build a larger library of accurate and diverse individual models to then construct ensembles. First, our extensive benchmarks show asynchronous Hyperband is an efficient and robust way to build a large number of diverse models to combine them. Then, a new ensemble selection method based on a multi-objective greedy algorithm is proposed to generate accurate ensembles by controlling their computing cost. Finally, we propose a novel algorithm to optimize the inference of the DNNs ensemble in a GPU cluster based on allocation optimization. The produced AutoML with ensemble method shows robust results on two datasets using efficiently GPU clusters during both the training phase and the inference phase. CCS CONCEPTS • Computing methodologies → Ensemble methods; Search methodologies; Massively parallel algorithms.
ACM___HPCASIA___AutoML_Ensemble_Inference.pdf (715.08 Ko) Télécharger le fichier

Dates et versions

hal-03517991 , version 1 (20-01-2022)

Identifiants

Citer

Pierrick Pochelu, Serge Petiton, Bruno Conche. A Deep Neural Networks ensemble workflow from hyperparameter search to inference leveraging GPU clusters. HPC Asia2022: International Conference on High Performance Computing in Asia-Pacific Region, Jan 2022, Virtual Event, Japan. pp.61-71, ⟨10.1145/3492805.3492819⟩. ⟨hal-03517991⟩
14 Consultations
102 Téléchargements

Altmetric

Partager

Gmail Facebook X LinkedIn More