In this article we present new experimental and theoretical results which were obtained for the flow between two concentric cylinders, with the inner one rotating and in the presence of an axial, stable density stratification. This system is characterized by two control parameters: one destabilizing, the rotation rate of the inner cylinder; and the other stabilizing, the stratification. Two oscillatory linear stability analyses assuming axisymmetric flow conditions are presented. First an eigenmode linear stability analysis is performed, using the small-gap approximation. The solutions obtained give insight into the instability mechanisms and indicate the existence of a confined internal gravity wave mode at the onset of instability. In the second stability analysis, only diffusion is neglected, predicting accurately the instability threshold as well as the critical pulsation for all the stratifications used in the experiments. Experiments show that the basic, purely azimuthal flow (circular Couette flow) is destabilized through a supercritical Hopf bifurcation to an oscillatory flow of confined internal gravity waves, in excellent agreement with the linear stability analysis. The secondary bifurcation, which takes the system to a pattern of drifting non-axisymmetric vortices, is a saddle-node bifurcation. The proposed bifurcation diagram shows a global bifurcation, and explains the discrepancies between previous experimental and numerical results. For slightly larger values of the rotation rate, weakly turbulent spectra are obtained, indicating an early appearance of weak turbulence: stationary structures and defects coexist. Moreover, in this regime, there is a large distribution of structure sizes. Visualizations of the next regime exhibit constant-wavelength structures and fluid exchange between neighbouring cells, similar to wavy vortices. Their existence is explained by a simple energy argument. The generalization of the bifurcation diagram to hydrodynamic systems with one destabilizing and one stabilizing control parameter is discussed. A qualitative argument is derived to discriminate between oscillatory and stationary onset of instability in the general case.