A Non-rigid Face Tracking Method for Wide Rotation Using Synthetic Data
Abstract
This paper propose a new method for wide-rotation non-rigid face tracking that is still a challenging problem in computer vision community. Our method consists of training and tracking phases. In training, we propose to use a large off-line synthetic database to overcome the problem of data collection. The local appearance models are then trained using linear Support Vector Machine (SVM). In tracking, we propose a two-step approach: (i) The first step uses baseline matching for a good initialization. The matching strategy between the current frame and a set of adaptive keyframes is also involved to be recoverable in terms of failed tracking. (ii) The second step estimates the model parameters using a heuristic method via pose-wise SVMs. The combination makes our approach work robustly with wide rotation, up to 90∘ of vertical axis. In addition, our method appears to be robust even in the presence of fast movements thanks to baseline matching. Compared to state-of-the-art methods, our method shows a good compromise of rigid and non-rigid parameter accuracies. This study gives a promising perspective because of the good results in terms of pose estimation (average error is less than 4o on BUFT dataset) and landmark tracking precision (5.8 pixel error compared to 6.8 of one state-of-the-art method on Talking Face video. These results highlight the potential of using synthetic data to track non-rigid face in unconstrained poses