Hand gesture recognition is recently becoming one of the most attractive field of research in Pattern Recognition. In this paper, a skeleton-based approach is proposed for 3D hand gesture recognition. Specifically, we consider the sequential data of hand geometric configuration to capture the hand shape variation, and explore the temporal character of hand motion. 3D Hand gesture are represented as a set of relevant spatiotemporal motion trajectories of hand-parts in an Euclidean space. Trajectories are then interpreted as elements lying on Riemannian manifold of shape space to capture their shape variations and achieve gesture recognition using a linear SVM classifier. The proposed approach is evaluated on a challenging hand gesture dataset containing 14 gestures, performed by 20 participants performing the same gesture with two di↵erent numbers of fingers. Experimental results show that our skeleton-based approach consistently achieves superior performance over a depth-based approach.