Deep Learning is nowadays widely used by security evaluators to conduct side-channel attacks, especially in profiling attacks that allow a supervised learning phase. However, designing an efficient neural network model in a side-channel attack context can be a difficult task that may require a laborious hyperparameterization process. Hyperparameter selection is known to be a challenging problem in Deep Learning, while being a crucial factor for neural networks performances. Recent works investigate the so-called Deep Ensemble Learning in the side-channel context. It consists in using multiple neural networks in a single predictive task and aggregating the several predictions in an opportune way. The intuition behind is to use the power of numbers to improve the attack performance. In this work, we propose to use Stacking as an aggregation method, in which a meta-model is trained to learn the best way to combine the output class probabilities of the ensemble networks. Our proposal is supported by several experimental results, that allow to conclude that the use of Stacking can relieve the security evaluator from performing a fine hyperparameterization.