The gravitational potential is a key function involved in many astrophysical problems. Its evaluation inside continuous media from Newton's law is known to be challenging because of the diverging kernel 1/|r-r'|. This difficulty is generally treated with avoidance techniques (e.g. multipole expansions, softening length) themselves not without drawbacks. In this article, we present a new path that basically fixes the point-mass singularity problem in systems with, at least, two dimensions. It consists of recasting the gravitational potential in an equivalent integro-differential form, namely the cross-derivative of a "hyperpotential" (i.e., an auxiliary scalar function). In contrast with the potential, the hyperpotential is the convolution of the mass density with a finite amplitude kernel. We show that closed-form expressions for this new kernel can be directly deduced from the potential of homogeneous sheets. We then give a few formulae appropriate to the Cartesian, cylindrical and spherical coordinate systems, including axial symmetry. The method is essentially not limited, either on the geometry of the source or on the distribution, and its implementation is straightforward. Several tests based upon simple quadrature/differentiation schemes are presented (the homogeneous rectangular sheet, cuboid and disk, the Maclaurin disk and a truncated Lane-Emden solution). Compared with a direct summation, the extra computational cost is low and the gain is real: no truncated series, no free parameter, and a relative accuracy better than 1% for typically 16 nodes per spatial direction using the most basic numerical schemes.