We study the convergence of the transport plans γε towards γ0 as well as the cost of the entropy-regularized optimal transport (c, γε) towards (c, γ0) as the regularization parameter ε vanishes in the setting of finite entropy marginals. We show that under the assumption of infinitesimally twisted cost and compactly supported marginals the distance W2(γε, γ0) is asymptotically greater than C √ ε and the suboptimality (c, γε) − (c, γ0) is of order ε. In the quadratic cost case the compactness assumption is relaxed into a moment of order 2 + δ assumption. Moreover, in the case of a Lipschitz transport map for the non-regularized problem, the distance W2(γε, γ0) converges to 0 at rate √ ε. Finally, if in addition the marginals have finite Fisher information, we prove (c, γε) − (c, γ0) ∼ dε/2 and we provide a companion expansion of H(γε). These results are achieved by disentangling the role of the cost and the entropy in the regularized problem.