Non-linear versions of log-Sobolev inequalities, that link a free energy to its dissipation along the corresponding Wasserstein gradient flow (i.e. corresponds to Polyak-Lojasiewicz inequalities in this context), are known to provide global exponential long-time convergence to the free energy minimizers, and have been shown to hold in various contexts. However they cannot hold when the free energy admits critical points which are not global minimizers, which is for instance the case of the granular media equation in a double-well potential with quadratic attractive interaction at low temperature. This work addresses such cases, extending the general arguments when a log-Sobolev inequality only holds locally and, as an example, establishing such local inequalities for the granular media equation with quadratic interaction either in the one-dimensional symmetric double-well case or in higher dimension in the low temperature regime. The method provides quantitative convergence rates for initial conditions in a Wasserstein ball around the stationary solutions. The same analysis is carried out for the kinetic counterpart of the gradient flow, i.e. the corresponding Vlasov-Fokker-Planck equation. The local exponential convergence to stationary solutions for the mean-field equations, both elliptic and kinetic, is shown to induce for the corresponding particle systems a fast (i.e. uniform in the number or particles) decay of the particle system free energy toward the level of the non-linear limit.