The spatial and spectral distributions of terahertz (THz) pulses emitted by two-color air plasmas are theoretically investigated for focused laser pulses and in the filamentation regime. We derive a so-called ''augmented'' conical emission model, which, similarly to the one originally proposed by You et al.\ [Phys.\ Rev.\ Lett.\ {\bf 109}, 183902 (2012)], involves phase matching between laser harmonics along the plasma channel, the plasma density and length, and the emitted frequency as key parameters. Our augmented model, however, treats envelope effects and accounts for transverse variations of both plasma- and Kerr-driven potential THz emitters. We highlight the importance of the characteristic spatio-spectral distributions of these two conversion mechanisms in the expression of the angular radiated power. The results of our model are successfully compared with data provided by a comprehensive, fully space and time-resolved unidirectional solver. Importantly, these numerical simulations clear up the effective plasma length along which THz emission develops, compared with the dephasing length along which the laser fundamental and second harmonic become out-of-phase. The impact of common optical aberrations, such as sphericity, astigmatism, and coma, on the THz generation is also investigated. Aberrations are shown to generally decrease the laser-to-THz conversion efficiency and potentially induce spatial asymmetries and narrowing in the THz spectra.