In this paper, we propose a novel human activity recognition approach based on STIPs' trajectories as local descriptors of video sequences. This representation compares favorably with state of art feature extraction methods. In addition, we investigate the use of SVM/HCRF combination for temporal sequence modeling, where SVM is applied locally on short video segments to produce probability scores, the latter being considered as the input vectors to HCRF. This method constitutes a new contribution to the state of the art on activity recognition task. The obtained results demonstrate that our method is efficient and compares favorably with state of the art methods on human activity recognition