In this paper we address the problem of generative object categorization in computer vision. We propose a Bayesian model using Hierarchical Dirichlet Processes mixing AdaBoost learning. Although previous methods trained HDP model for one or two latent themes, our proposed approach uses small-patch-independent-words of appearance-based descriptor and shape information to train a set of intermediate components which are the mixture of visualwords. We then employ AdaBoost weaker learner to find the most related components for classification to handle the variance in intraclass and inter-class information. We show that it performs well for Caltech datasets and with the potential to connect the visual concepts with semantic concepts.