In high resolution multispectral optical data, the spatial detail of the images are generally smaller than the dimensions of objects, and often the spectral signature of pixels is not directly representative of classes we are interested in. Thus, taking into account the relations between groups of pixels becomes increasingly important, making object­oriented approaches preferable. In this work several scales of detail within an image are considered through a hierarchical segmentation approach, while the spectral information content of each pixel is accounted for by a per­pixel classification. The selection of the most suitable spatial scale for each class is obtained by merging the hierarchical segmentation and the per­pixel classification through the Scale Object Selection (SOS) algorithm. The SOS algorithm starts processing data from the highest level of the hierarchical segmentation, which has the least amount of spatial detail, down to the last segmentation map. At each segmentation level, objects are assigned to a specific class whenever the percentage of pixels belonging to the latter, according to a pixel­based procedure, exceeds a predefined threshold, thereby automatically selecting the most appropriate spatial scale for the classification of each object. We apply our method to multispectral, panchromatic and pan­sharpened QuickBird images.