We study the moduli space of trace-free irreducible rank 2 connections over a curve of genus 2 and the forgetful map towards the (non-separated) moduli space of underlying vector bundles (including unstable bundles). We draw the geometric picture of the latter and compute a natural Lagrangian rational section of the forgetful map. As a particularity of the genus 2 case, such connections are invariant under the hyperelliptic involution : they descend as rank 2 logarithmic connections over the Riemann sphere. We establish explicit links between the well-known moduli space of the underlying parabolic bundles with the classical approaches by Narasimhan-Ramanan, Tyurin and Bertram. By the hyperelliptic approach, we recover a Poincaré family on a degree 2 cover of the Narasimhan-Ramanan moduli space, due to Bolognesi. Moreover, we compare the explicit equations of the Kummer surface and the Hitchin map for each point of view, allowing us to explain a certain number of geometric phenomena in the considered moduli spaces.