The multi-scale hybridization of carbon nanotubes (CNTs) with micro-particles in polymers offers new opportunity to develop high performance multifunctional composites. In this study, hybrid fillers comprised of CNTs directly grown on alumina micro-spheres by chemical vapor deposition were incorporated into epoxy matrix that was then reinforced with woven glass fibers. The hierarchical composites with 0.5 wt.% hybrid loading was observed to exhibit an improvement of 19% and 11% in flexural modulus and interlaminar shear strength, respectively. Moreover, the glass transition temperature was increased by 15 °C and the storage modulus at 50 °C was enhanced by 20%. These reinforcements are mainly attributed to the improvements of matrix properties resulted from the good dispersion of hybrids and their hindering effect on the formation and development of matrix cracks. This study reveals the potential in improving mechanical and thermo-mechanical properties of the fiber-reinforced composite by using multi-scale carbon hybrids.