We establish small-time asymptotic expansions for heat kernels of hypoelliptic Hörmander operators in a neighborhood of the diagonal, generalizing former results obtained in particular by Métivier and by Ben Arous. The coefficients of our expansions are identified in terms of the nilpotentization of the underlying sub-Riemannian structure. Our approach is purely analytic and relies in particular on local and global subelliptic estimates as well as on the local nature of small-time asymptotics of heat kernels. The fact that our expansions are valid not only along the diagonal but in an asymptotic neighborhood of the diagonal is the main novelty, useful in view of deriving Weyl laws for subelliptic Laplacians. In turn, we establish a number of other results on hypoelliptic heat kernels that are interesting in themselves, such as Kac's principle of not feeling the boundary, asymptotic results for singular perturbations of hypoelliptic operators, global smoothing properties for selfadjoint heat semigroups.