The chemical selectivity and great sensitivity of the Extended X-ray Absorption Spectroscopy technique allowed the determination, in the paramagnetic phase, of the structural distortions induced by doping in the spin-Peierls CuGeO$_3$ compound. The distorted environments were analyzed as a function of concentration, magnetic nature of impurity and the substitution site (Ni, Mn and Zn impurities on the Cu site, Si impurity on the Ge site). This has led to estimate the variation of the angles and pair distances, and hence to evaluate the magnetic coupling along the Cu chains in the vicinity of the impurities. The antiferromagnetic interaction between Cu first neighbors in the pure sample is found to be weakened around Ni, almost cancelled in the case of Mn doping, and even to change sign, producing a ferromagnetic coupling for Si doping. More generally, the structural distortions on a local scale are shown to be key parameters for the understanding of the magnetic properties of doped spin-Peierls compounds.