By using various properties of the complete elliptic integrals, we have derived an alternative expression for the gravitational potential of axially symmetric bodies, which is free of singular kernel in contrast with the classical form. This is mainly a radial integral of the local surface density weighted by a regular "mean Green function" which depends explicitly on the body's vertical thickness. Rigorously, this result stands for a wide variety of configurations, as soon as the density structure is vertically homogeneous. Nevertheless, the sensitivity to vertical stratification | the Gaussian profile has been considered | appears weak provided that the surface density is conserved. For bodies with small aspect ratio (i.e. geometrically thin discs), a first-order Taylor expansion furnishes an excellent approximation for this mean Green function, the absolute error being of the fourth order in the aspect ratio. This formula is therefore well suited to studying the structure of self-gravitating discs and rings in the spirit of the "standard model of thin discs" where the vertical structure is often ignored, but it remains accurate for discs and tori of finite thickness. This approximation which perfectly saves the properties of Newton's law everywhere (in particular at large separations), is also very useful for dynamical studies where the body is just a source of gravity acting on external test particles.