Reliable obstacle detection and localization is a key issue for driving assistance systems particularly in urban environments. In this article, a multi-modal perception approach is studied in order to enhance vehicle localization and dynamic objects tracking, in a world-centric map. 3D ego-localization is done by merging a stereo vision system and proprioceptive information coming from vehicle sensors. Mobile objects are detected using a multi-layer lidar that is simultaneously used to characterize a zone of interest in order to reduce the complexity of the perception process. Objects localization and tracking is then performed in the fixed frame which simplifies the scene analysis and understanding. Real experimental results are reported to evaluate the performance of the multi-modal system.