Recognition of facial action units (AU) is one of two main streams in the facial expressions analysis. Action units deform facial appearance simultaneously in landmark locations and local texture as well as geometry on 3D faces. Thus, it is necessary to use features extracted from multiple facial modalities to characterize these deformations comprehensively. In order to fuse the contribution of the discriminative power from all features efficiently, we propose to use our extended statistical facial feature models (SFAM) to generate feature instances corresponding to AU class for each feature. Then the similarity between each feature on a face and its instances are evaluated so that a set of similarity scores are obtained. All sets of scores on the face are then weighted for AU recognition. Experiments on the Bosphorus database show its state-of-the- art performance.