Visual content description is a key issue for machine-based visual object recognition, which is one of the most challenging problems in computer vision, due to intra-class variations and inter-class similarities. A good visual descriptor should be discriminative enough and computationally efficient while displaying some properties of robustness to variations. The recent literature has featured local appearance-based features, e.g. SIFT, as the main trend because of their great discriminative power. In this paper, we propose to adopt two different kinds of feature to characterize different aspects of object appearance. The first is multi-scale local binary pattern (LBP) operator, which is extracted from coarse-to-fine image blocks to well describe texture structures while keeping its computational efficiency. The second is line segment feature, which is based on Gestalt-inspired region segmentation and fast Hough transform, and aims at capturing accurate geometric information of visual objects. The experimental results on the SIMPLIcity database and PASCAL VOC 2007 benchmark show the effectiveness of line segment feature, and significant accuracy improvement by using fine level image blocks for LBP. Moreover, combining LBP from different image block levels can further boost its performance, and outperform the state-of-the-art SIFT. Both descriptors are also proven to provide complementary information to the SIFT.