Reliable obstacle detection and localization is a key issue for driver assistance systems, particularly in urban environments. In this study a multi-modal perception approach is investigated, the objective being to enhance vehicle localization and dynamic object tracking in a world-centric map. A 3D ego-localization is achieved by merging information from a stereo vision system and data obtained from vehicle sensors. Mobile objects are detected using a multi-layer lidar that is also used to identify a constrained search space within the multiple target tracking process. Object localization and tracking is then performed in the fixed frame, which facilitates analysis and understanding of the scene. Experimental results using real world data are performed to evaluate the performance of the multi-modal system, and these are presented to show the effectiveness of the approach.