Symmetric Positive Definite (SPD) matrices have been usedin many fields of medical data analysis. Many Riemannian metrics havebeen defined on this manifold but the choice of the Riemannianstructurelacks a set of principles that could lead one to choose properly the met-ric. This drives us to introduce the principle of balanced metrics that re-late the affine-invariant metric with the Euclidean and inverse-Euclideanmetric, or the Bogoliubov-Kubo-Mori metric with the Euclidean and log-Euclidean metrics. We introduce two new families of balanced metrics,the mixed-power-Euclidean and the mixed-power-affine metrics and wediscuss the relation between this new principle of balanced metrics and the concept of dual connection in information geometry.