We study the moduli space of trace-free irreducible rank 2 connections over a curve of genus 2 and the forgetful map towards the moduli space of under- lying vector bundles (including unstable bundles), for which we compute a natural Lagrangian rational section. As a particularity of the genus 2 case, connections as above are invariant under the hyperelliptic involution : they descend as rank 2 logarithmic connections over the Riemann sphere. We establish explicit links between the well-known moduli space of the underlying parabolic bundles with the classical approaches by Narasimhan-Ramanan, Tyurin and Bertram. This allow us to explain a certain number of geometric phenomena in the considered moduli spaces such as the classical (16, 6)-configuration of the Kummer surface. We also recover a Poincaré family due to Bolognesi on a degree 2 cover of the Narasimhan-Ramanan moduli space. We explicitly compute the Hitchin integrable system on the moduli space of Higgs bundles and compare the Hitchin Hamiltonians with those found by vanGeemen-Previato. We explicitly describe the isomonodromic foliation in the moduli space of vector bundles with sl(2,C)-connection over curves of genus 2 and prove the transversality of the induced flow with the locus of unstable bundles.