Hybrid organic-inorganic perovskites emerged as a new generation of absorber materials for high-efficiency low-cost solar cells in 2009. Very recently, fully inorganic perovskite quantum dots also led to promising efficiencies, making them a potentially stable and efficient alternative to their hybrid cousins. Currently, the record efficiency is obtained with CsPbI3 whose crystallographical characterization is still limited. Here we show through high resolution in-situ synchrotron XRD measurements that CsPbI3 can be undercooled below its transition temperature and temporarily maintained in its perovskite structure down to room temperature, stabilizing a metastable perovskite polytype (black γ-phase) crucial for photovoltaic applications. Our analysis of the structural phase transitions reveals a highly anisotropic evolution of the individual lattice parameters versus temperature. Structural, vibrational and electronic properties of all the experimentally observed black phases are further inspected based on several theoretical approaches. While the black γ-phase is shown to behave harmonically around equilibrium, for the tetragonal phase density functional theory reveals the same anharmonic behavior, with a Brillouin zone-centered double-well instability, as for the cubic phase. Using total energy and vibrational entropy calculations, we highlight the competition between all the low-temperature phases of CsPbI3 (γ, δ, β) and show that avoiding the order-disorder entropy term arising from double-well instabilities is key in order to prevent the formation of the yellow perovskitoid phase. A symmetry-based tight-binding model, validated by self-consistent GW calculations including spin-orbit coupling, affords further insight into their electronic properties, with evidence of Rashba effect for both cubic and tetragonal phases when using the symmetry breaking structures obtained through frozen phonon calculations.