The use of ontology is common for many domains such as semantic web, e-commerce, artificial intelligence and geographical information systems. Besides, the vocabulary used in ontologies is always textual; concepts and relations are identified and labelled by words. However, some concepts include a visual aspect especially in the cartographic domain. For example, in a cartographic legend, a point of interest symbol is identified as a concept with its icon and/or abbreviation, color, texture, font style, orientation or number. The overall concerns of our research are to integrate different legends as visual ontologies towards a unique reference one (base map and symbols from many providers). The objective of our article is to 1) propose a new type of ontology where the concepts are visual and to apply it to cartographic symbols then to 2) develop an application prototype in order to handle these kinds of ontologies with an extension of Web Ontology Language (CartOWL).