In this paper, we investigate the minimal time problem for the guidance of a rocket, whose motion is described by its attitude kinematics and dynamics but also by its orbit dynamics. Our approach is based on a refined geometric study of the extremals coming from the application of the Pontryagin maximum principle. Our analysis reveals the existence of singular arcs of higher-order in the optimal synthesis, causing the occurrence of a chattering phenomenon, i.e., of an infinite number of switchings when trying to connect bang arcs with a singular arc. We establish a general result for bi-input control-affine systems, providing sufficient conditions under which the chattering phenomenon occurs. We show how this result can be applied to the problem of the guidance of the rocket. Based on this preliminary theoretical analysis, we implement efficient direct and indirect numerical methods, combined with numerical continuation, in order to compute numerically the optimal solutions of the problem.