This work aims to identify abnormal behaviors from the analysis of humans or vehicles' trajectories. A set of normal trajectories' prototypes is extracted by means of a novel unsupervised learning technique: the scene is adaptively partitioned into zones by using the distribution of the training set and each trajectory is represented as a sequence of symbols by taking into account positional information (the zones crossed in the scene), speed and shape. The main novelties of this work are the following: first, the similarity between trajectories is evaluated by means of a kernel-based approach. Furthermore, we define a novel and efficient kernel-based clustering algorithm, aimed at obtaining groups of normal trajectories. The proposed approach has been compared with state-of-the-art methods and it clearly outperforms all the other considered techniques.