In this paper, we propose an improved image retrieval method, dedicated to images of buildings/landmarks from urban environments. Locally detected key points are binary labelled as building or no-building using a SVM-based classifier. Thereafter, only key points labelled as building are retained. In this way, the data in the database vocabulary is reduced to only the relevant one and solely the relevant features, effectively describing the targeted buildings are considered. The experimental results, carried out on the Paris6k and Oxford5k data sets show significant improvement in terms of retrieval precision