Estimating the cardiac motion from ultrasound (US) images is an ill-posed problem that requires regularization. In a recent study, it was shown that constraining the cardiac motion fields to be patch-wise sparse in a learnt overcomplete motion dictionary is more accurate than local parametric models (affine) or global functions (B-splines, total variation). In this work, we extend this method by incorporating temporal smoothness in a multi-frame optical-flow (OF) strategy. An efficient optimization strategy using the constrained split augmented Lagrangian shrinkage algorithm (C-SALSA) is proposed. The performance is evaluated on a realistic simulated cardiac dataset with available ground-truth. A comparison with the pairwise approach shows the interest of the proposed temporal regularization and multi-frame strategy in terms of accuracy and computational time.