This paper considers the problem of obtaining a low-rank factorization of a given correlation matrix. In order to handle possible spurious correlation coefficients within the input, a robust formulation is proposed with a criterion based on the Huber loss function. Minimizing this fitting criterion under the low-rank correlation structure constraint is then addressed using the block majorization-minimization framework. Several algorithm options are explored and compared in terms of computational complexity. The merits of the proposed correlation fitting method are then validated on simulations, and for the process of dimension reduction of microarray data.