In the context of digital in-line holographic microscopy, we describe an unsupervised methodology to estimate the aberrations of an optical microscopy system from a single hologram. The method is based on the Inverse Problems Approach reconstructions of holograms of spherical objects. The forward model is based on a Lorenz-Mie model distorted by optical aberrations described by Zernike polynomials. This methodology is thus able to characterize most varying aberrations in the field of view in order to take them into account to improve the reconstruction of any sample. We show that this approach increases the repeatability and quantitativity of the reconstructions in both simulations and experimental data. We use the Cramér-Rao lower bounds to study the accuracy of the reconstructions. Finally, we demonstrate the efficiency of this aberration calibration with image reconstructions using a phase retrieval algorithm as well as a regularized inverse problems algorithm.