Heralded as one of the key elements for next generation spintronics devices, topological insulators (TIs) are now step by step envisioned as nanodevices like charge-to-spin current conversion or as Dirac fermions based nanometer Schottky diode for example. However, reduced to few nanometers, TIs layers exhibit a profound modification of the electronic structure and the consequence of this quantum size effect on the fundamental carriers and phonons ultrafast dynamics has been poorly investigated so far. Here, thanks to a complete study of a set of high quality molecular beam epitaxy grown nanolayers, we report the existence of a critical thickness of around ~6 nm, below which a spectacular reduction of the carrier relaxation time by a factor of ten is found in comparison to bulk Bi2 Te3 In addition, we also evidence an A1g optical phonon mode softening together with the appearance of a thickness dependence of the photoinduced coherent acoustic phonons signals. This drastic evolution of the carriers and phonons dynamics might be due an important electron-phonon coupling evolution due to the quantum confinement. These properties have to be taken into account for future TIs-based spintronic devices.