The increasing number and resolution of earth observation (EO) imaging sensors has had a significant impact on both the acquired image data volume and the information content in images. There is consequently a strong need for highly efficient search tools for EO image databases and for search methods to automatically identify and recognize structures within EO images. Content Based Image Retrieval (CBIR) and automatic image annotation systems have been designed to tackle the problem of image retrieval in large image databases. These two systems achieve a common goal which is to learn the mapping function between low-level visual features and high-level image semantics. In this paper, we present an overview of two approaches that address the problem of learning this mapping function from a few training examples in the case of auto-annotation and CBIR systems respectively.