In this paper, we consider the problem of tracking the signal subspace under a sparsity constraint over the weight basis matrix. In the same spirit of our previous work [1, 2], we propose a new low cost algorithm which has at every iteration two stages. First, it estimates adaptively an orthogonal basis for the signal subspace. Then, it uses the Shear and Givens rotations to minimize the sparsity criterion. The proposed algorithm has a approximately the same performance compared to our previous best proposition in [2]. However, the new algorithm has a low (linear) computational complexity. Moreover, simulation results show that it outperforms the state of art solutions.