We report on a vertically scaled AlN/GaN HEMT technology design optimization for millimeter-wave applications. The undoped GaN channel thickness and carbon concentration into the buffer are extensively varied and systematically characterized. It is found that a thin GaN channel, typically below 150 nm improves the electron confinement, but increases the trapping effects, especially when using shorter gate lengths. Moreover, high carbon concentration into the buffer enables not only high electron confinement but also low leakage current under a high electric field at the expense of trapping effects. As a result, the optimum epi-design enabled state-of-the-art RF performances at 40 GHz.