The simulation of heterogeneous Virtual Humans needs a lot of structured information: geometric information for animation and semantic information for behavioral simulation. This complexity needs a robust structure to represent the whole of necessary data. We propose to use ontology as a tool for design the data structure. Our model provides a modeling of the following parts: the virtual environment, its structure, the knowledge items using ontology, interactions and tasks that virtual human. In our model, we find two types of ontology: a global ontology and the local ontologies dedicated to each Virtual Human.