Structure–property correlations are often hampered by insufficient structural insight into the crystal packing of polymer semiconductors widely used in electronic devices such as organic solar cells. Herein, both the semicrystalline morphology and the crystalline structures of two high performance polymer semiconductors showing more than 9% efficiencies, namely PDTBT-TT and PBT4T, are established by a combination of oriented crystallization and transmission electron microscopy. PDTBT-TT and PBT4T form layered structures with alternation of π-stacked backbones and layers of disordered alkyl side chains. π-Stacking is such that benzothiadiazole and the comonomer (quaterthiophene or thienothiophene–bithiophene) segregate to form distinct stacks. This segregated stacking is preferentially obtained in thin films aligned by high-temperature rubbing at T = 200–230 °C. However, the two polymers show different stabilities of this polymorph versus temperature. The segregated stacking of PDTBT-TT is stable up to near the melting temperature, whereas for PBT4T it transforms to a layered structure with significant intrastack disorder at T ≥ 250 °C. The intensity of the 0–0 component of the vibronic progression in the absorption spectrum is enhanced for the polymorph with long-range segregated π-stacks. The structural models determined for the two polymers suggest that both the position of alkyl side chains and the preferential π-stacking interactions between comonomers determine the polymorphism and corresponding thermal stability.