Speech emotion is high semantic information and its automatic analysis may have many applications such as smart human-computer interactions or multimedia indexing. Among the main difficulties encountered when developing an automatic recognition system are the definition of emotions and their subjective nature. Thus, with the consideration of a limited number of types of emotions, a certain emotional state can be between some pre-defined emotional states, while a too large number of types can lead to an insolvable classification problem. In this case, a classification with the management of ambiguous emotions is necessary. Such an approach is proposed in this paper: in order to make the automatic recognition of emotions as close as possible to the judgments produced by humans, we developed an ambiguous classifier which allows to give multiple labels to emotional speech. This approach has been evaluated on Berlin dataset and compared with multiple human judgments used as ground truth.