Advanced Side-Channel Analyses make use of dimensionality reduction techniques to reduce both the memory and timing complexity of the attacks. The most popular methods to effectuate such a reduction are the Principal Component Analysis (PCA) and the Linear Discrim-inant Analysis (LDA). They indeed lead to remarkable efficiency gains but their use in side-channel context also raised some issues. The PCA provides a set of vectors (the principal components) onto which project the data. The open question is which of these principal components are the most suitable for side-channel attacks. The LDA has been valorized for its theoretical leaning toward the class-distinguishability, but discouraged for its constraining greed of data. In this paper we present an in-depth study of these two methods, and, to automatize and to ameliorate the principal components selection, we propose a new technique named cumulative Explained Local Variance (ELV) selection. Moreover we present some extensions of the LDA, available in less constrained situations than the classical version. We equip our study with a comprehensive comparison of the existing and new methods in real cases. It allows us to verify the soundness of the ELV selection, and the effectiveness of the methods proposed to extend the use of the LDA to side-channel contexts where the existing approaches are inapplicable.