In this paper we pursue the work initiated in \cite{Bahuaud, BahuaudGicquaud}: study the extent to which conformally compact asymptotically hyperbolic metrics can be characterized intrinsically. We show how the decay rate of the sectional curvature to -1 controls the Hölder regularity of the compactified metric. To this end, we construct harmonic coordinates that satisfy some Neumann-type condition at infinity. Combined with a new integration argument, this permits us to recover to a large extent our previous result without any decay assumption on the covariant derivatives of the Riemann tensor. We believe that our result is optimal.