In this paper we introduce and rigorously solve for the first time the photometric stereo problem for central panoramic systems. In particular, by leveraging the unified camera model, we present a new spherical image irradiance equation and analyze its mathematical properties in detail. The discrete gradient field issued from this equation is fed into a drift-free normal integration algorithm tailored to the spherical image geometry, and the light direction is estimated using the specular highlights observed on mirror balls inside the scene. Extensive experiments conducted with analytical surfaces, and synthetic and real-world images captured by central panoramic cameras, show that the proposed 3D reconstruction pipeline is effective and tolerant to noise. The image database and the code developed are publicly available at the address: mis.u-picardie.fr/˜fabio/ustereo.html