Spatio-Temporal Convolutional Sparse Auto-Encoder for Sequence Classification - Archive ouverte HAL Access content directly
Conference Papers Year : 2012

Spatio-Temporal Convolutional Sparse Auto-Encoder for Sequence Classification


We present in this paper a novel learning-based approach for video sequence classification. Contrary to the dominant methodology, which relies on hand-crafted features that are manually engineered to be optimal for a specific task, our neural model automatically learns a sparse shift-invariant representation of the local 2D+t salient information, without any use of prior knowledge. To that aim, a spatio-temporal convolutional sparse auto-encoder is trained to project a given input in a feature space, and to reconstruct it from its projection coordinates. Learning is performed in an unsupervised manner by minimizing a global parametrized objective function. The sparsity is ensured by adding a sparsifying logistic between the encoder and the decoder, while the shift-invariance is handled by including an additional hidden variable to the objective function. The temporal evolution of the obtained sparse features is learned by a long short-term memory recurrent neural network trained to classify each sequence. We show that, since the feature learning process is problem-independent, the model achieves outstanding performances when applied to two different problems, namely human action and facial expression recognition. Obtained results are superior to the state of the art on the GEMEP-FERA dataset and among the very best on the KTH dataset.

Dates and versions

hal-01353046 , version 1 (10-08-2016)



Moez Baccouche, Franck Mamalet, Christian Wolf, Christophe Garcia, Atilla Baskurt. Spatio-Temporal Convolutional Sparse Auto-Encoder for Sequence Classification. British Machine Vision Conference (BMVC), Sep 2012, Guildford, United Kingdom. pp.124.1-124.12, ⟨10.5244/C.26.124⟩. ⟨hal-01353046⟩
279 View
0 Download



Gmail Facebook Twitter LinkedIn More