Since the challenging visual object categorization has attracted more and more attention in recent years, we present in this paper a novel approach called statistical measures based image modeling for this problem, thus avoiding the major difficulty of the popular “bag-of-visual words” approach which needs to fix a visual vocabulary size. We use a series of statistical measures over our proper region based color and segment features as well as the popular SIFT, extracted from an image, to model its visual content. Then this new image modeling will be fed to a certain classifier to accomplish the object categorization task. Several classification schemes combined with some feature selection techniques and fusion strategies have also been implemented and compared within the experimentation carried out on a subset of Pascal VOC dataset. The results show that merging the region based features and SIFT which are from different sources using an early fusion can actually improve classification performance, suggesting that these features managed to extract information which is complementary to each other.