In this paper we present a face model based on learning a relation between local features and a face invariant. We have developed a face invariant model for accurate face localization in natural images that is robust to viewpoints changes. A probabilistic model learned from a training set, captures a relationship between features appearance and face invariant geometry. It is then used to infer a face instance in new images. We use the invariant local features which have a high performance for objects appearance distinctiveness. The face appearance features are recognized by EM classification. Then, face invariant parameters are predicted and a hierarchical clustering method achieves invariant geometric localization. The clustering uses an aggregate value to construct clusters of invariants. The face appearance probabilities of features are computed to select the best clusters and thus to localize faces in images. We evaluate our generic invariant by testing it in face detection experiments on PIE, FERET and CMU-Profiles databases. The experimental results show that our face invariant model gives highly accurate face localization.