All inorganic CsPbX3 (X = Cl, Br, I) nanocrystals (NCs) belong to the novel class of confined metal–halide perovskites which are currently arousing enthusiasm and stimulating huge activity across several fields of optoelectronics due to outstanding properties. A deep knowledge of the band-edge excitonic properties of these materials is thus crucial to further optimize their performances. Here, high-resolution photoluminescence (PL) spectroscopy of single bromide-based NCs reveals the exciton fine structure in the form of sharp peaks that are linearly polarized and grouped in doublets or triplets, which directly mirror the adopted crystalline structure, tetragonal (D4h symmetry) or orthorhombic (D2h symmetry). Intelligible equations are found that show how the fundamental parameters (spin–orbit coupling, ΔSO, crystal field term, T, and electron–hole exchange energy, J) rule the energy spacings in doublets and triplets. From experimental data, fine estimations of each parameter are obtained. The analysis of the absorption spectra of an ensemble of NCs with a “quasi-bulk” behavior leads to ΔSO = 1.20 ± 0.06 eV and T = −0.34 ± 0.05 eV in CsPbBr3. The study of individual luminescence responses of NCs having sizes comparable to the exciton Bohr diameter, 7 nm, allows us to estimate the value of J to be around ≈3 meV in both tetragonal and orthorhombic phases. This value is already enhanced by confinement.