In this paper, we consider supervised learning problems such as logistic regression and study the stochastic gradient method with averaging, in the usual stochastic approximation setting where observations are used only once. We show that for self-concordant loss functions, after $n$ iterations, with a constant step-size proportional to $1/R^2 \sqrt{n}$ where $n$ is the number of observations and $R$ is the maximum norm of the observations, the convergence rate is always of order $O(1/\sqrt{n})$, and improves to $O(R^2 / \mu n)$ where $\mu$ is the lowest eigenvalue of the Hessian at the global optimum (when this eigenvalue is strictly positive). Since $\mu$ does not need to be known in advance, this shows that averaged stochastic gradient is adaptive to unknown local strong convexity of the objective function.