The extended use of high and very high spatial resolution imagery inherently demands the adoption of classification methods capable of capturing the underlying semantic. Object-oriented classification methods are currently considered as the most appropriate alternative, due to the incorporation of contextual information and domain knowledge into the analysis. Integrating knowledge initially requires a detailed process of acquisition and later the achievement of a formal representation. Ontologies constitute a very suitable approach to address both knowledge formalization and exploitation. A novel semi-automatic fuzzy semantic approach focused on the extraction and classification of urban objects is hereby introduced. The use of a four-layered architecture allows the separation of concerns among knowledge, rules, experience and meta-knowledge. Knowledge represents the fundamental layer with which the other layers interact. Rules are meant to derive conclusions and make assertions based on knowledge. The experience layer supports the classification process in case of failure when attempting to identify an object, by applying specific expert rules to infer unusual membership. Finally, the meta-knowledge layer contains knowledge about the use of the other layers.