VANET aims to improve safety for all road users. Vehicles exchange safety messages over wireless communication links which are prone to multiple attacks. To enhance the existing security of V2V communications, we propose in this paper a security framework based on vehicles behavior analysis. We define a Hybrid Trust Model (HTM) and a misbehavior detection system (MDS) where a trust metric is assigned to every vehicle depending on its behavior. Using this trust metric, a classification of the vehicles into malicious or honest is done. HTM is based on-the-fly group formation which helps to manage the communication between vehicles and the back-end system by selecting the most trustworthy node as group leader (GL). Vehicles and GL will cooperate with each other within the group and with the back-end system to detect the malicious node and to notify the Misbehavior Authority. The latter takes appropriate actions to limit the consequences of the malicious behaving node. Performance evaluation of HTM and MDS is carried out using Groovenet Simulator. Results show the efficiency of the proposed model to select the trustworthy vehicles and to monitor their behaviors, as well as to classify them and to deactivate the malicious ones.