The stability properties of a vortex lens are studied in the quasi geostrophic (QG) framework using the generalized stability theory. Optimal perturbations are obtained using a tangent linear QG model and its adjoint. Their fine-scale spatial structures are studied in details. Growth rates of optimal perturbations are shown to be extremely sensitive to the time interval of optimization: The most unstable perturbations are found for time intervals of about 3 days, while the growth rates continuously decrease towards the most unstable normal mode, which is reached after about 170 days. The horizontal structure of the optimal perturbations consists of an intense counter-shear spiralling. It is also extremely sensitive to time interval: for short time intervals, the optimal perturbations are made of a broad spectrum of high azimuthal wave numbers. As the time interval increases, only low azimuthal wave numbers are found. The vertical structures of optimal perturbations exhibit strong layering associated with high vertical wave numbers whatever the time interval. However, the latter parameter plays an important role in the width of the vertical spectrum of the perturbation: short time interval perturbations have a narrow vertical spectrum while long time interval perturbations show a broad range of vertical scales. Optimal perturbations were set as initial perturbations of the vortex lens in a fully non linear QG model. It appears that for short time intervals, the perturbations decay after an initial transient growth, while for longer time intervals, the optimal perturbation keeps on growing, quickly leading to a non-linear regime or exciting lower azimuthal modes, consistent with normal mode instability. Very long time intervals simply behave like the most unstable normal mode. The possible impact of optimal perturbations on layering is also discussed