Estimation procedures based on recursive algorithms are interesting and powerful techniques that are able to deal rapidly with (very) large samples of high dimensional data. The collected data may be contaminated by noise so that robust location indicators, such as the geometric median, may be preferred to the mean. In this context, an estimator of the geometric median based on a fast and efficient averaged non linear stochastic gradient algorithm has been developed by Cardot et al. (2013). This work aims at studyingmore precisely the non asymptotic behavior of this algorithmby giving non asymptotic confidence balls. This newresult is based on the derivation of improved L2 rates of convergence as well as an exponential inequality for the martingale terms of the recursive non linear Robbins-Monro algorithm.