Fish-Eye image processing with conventional Machine Learning algorithms such as Convolutional Neural Networks is a challenging task because of the distortion effects induced by dewarping the raw hemispherical images into the Euclidean plane. We introduce in this paper an approach based on the emerging field of Geometric Deep Learning in which hyperbolic projection techniques are coupled with equivariance mechanisms in order to preserve native geometrical dependencies and to achieve robustness with respect to natural variations in the perception of the Fish-Eye images. This work in particular motivates the development of efficient SU(1, 1) and SL(2, R) Equivariant Neural Networks.