We study the non-asymptotic behavior of Coulomb gases in dimension two and more. Such gases are modeled by an exchangeable Boltzmann-Gibbs measure with a singular two-body interaction. We obtain concentration of measure inequalities for the empirical distribution of such gases around their equilibrium measure, with respect to bounded Lipschitz and Wasserstein distances. This implies macroscopic as well as mesoscopic convergence in such distances. In particular, we improve the concentration inequalities known for the empirical spectral distribution of Ginibre random matrices. Our approach is remarkably simple and bypasses the use of renormalized energy. It crucially relies on new inequalities between probability metrics, including Coulomb transport inequalities which can be of independent interest. Our work is inspired by the one of Maïda and Maurel-Segala, itself inspired by large deviations techniques. Our approach allows to recover, extend, and simplify previous results by Rougerie and Serfaty.