Many signal and image processing applications, including texture analysis, radar detection or EEG signal classification, require the computation of a centroid from a set of covariance matrices. The most popular approach consists in considering the center of mass. While efficient, this estimator is not robust to outliers arising from the inherent variability of the data or from faulty measurements. To overcome this, some authors have proposed to use the median as a more robust estimator. Here, we propose an estimator which takes advantage of both efficiency and robustness by combining the concepts of Riemannian center of mass and median. Based on the theory of M-estimators, this robust centroid estimator is issued from the so-called Huber's function. We present a gradient descent algorithm to estimate it. In addition, an experiment on both simulated and real data is carried out to evaluate the influence of outliers on the estimation and classification performances.