Attaining low-resistivity Al$_x$Ga$_{1–x}$N layers is one keystone to improve the efficiency of light-emitting devices in the ultraviolet spectral range. Here, we present a microstructural analysis of Al$_x$Ga$_{1–x}$N/Ge samples with 0 ≤ x ≤ 1, and a nominal doping level in the range of 10$^{20}$ cm$^{–3}$, together with the measurement of Ge concentration and its spatial distribution down to the nanometer scale. Al$_x$Ga$_{1–x}$N/Ge samples with x ≤ 0.2 do not present any sign of inhomogeneity. However, samples with x > 0.4 display μm-size Ge crystallites at the surface. Ge segregation is not restricted to the surface: Ge-rich regions with a size of tens of nanometers are observed inside the Al$_x$Ga$_{1–x}$N/Ge layers, generally associated with Ga-rich regions around structural defects. With these local exceptions, the Al$_x$Ga$_{1–x}$N/Ge matrix presents a homogeneous Ge composition which can be significantly lower than the nominal doping level. Precise measurements of Ge in the matrix provide a view of the solubility diagram of Ge in Al$_x$Ga$_{1–x}$N as a function of the Al mole fraction. The solubility of Ge in AlN is extremely low. Between AlN and GaN, the solubility increases linearly with the Ga mole fraction in the ternary alloy, which suggests that the Ge incorporation takes place by substitution of Ga atoms only. The maximum percentage of Ga sites occupied by Ge saturates around 1%. The solubility issues and Ge segregation phenomena at different length scales likely play a role in the efficiency of Ge as an n-type AlGaN dopant, even at Al concentrations where Ge DX centers are not expected to manifest. Therefore, this information can have direct impact on the performance of Ge-doped AlGaN light-emitting diodes, particularly in the spectral range for disinfection (≈260 nm), which requires heavily doped alloys with a high Al mole fraction.