Recent developments with indefinite SVM [11, 17, 5] have effectively demonstrated SVM classification with a non-positive kernel. However the question of efficiency still applies. In this paper, an efficient direct solver for SVM with non-positive kernel is proposed. The chosen approach is related to existing work on learning with kernel in Krein space. In this framework, it is shown that solving a learning problem is actually a problem of stabilization of the cost function instead of a minimization. We propose to restate SVM with non-positive kernels as a stabilization by using a new formulation of the KKT conditions. This new formulation provides a practical active set algorithm to solve the indefinite SVM problem. We also demonstrate empirically that the proposed algorithm outperforms other existing solvers.