In this paper, we propose a multi-sensors system to detect people in the context of construction sites using heavy machines. The system includes a LIght Detection And Ranging (Lidar) sensor and a fisheye camera. We present an effective method to determine regions of interest (ROI) on fisheye images using Lidar data, which can be used with different sensors configurations. A Deformable Part Model (DPM) approach is adapted and used as the main people detector on image. We also present a specific dataset built using the multi-sensor system mounted on a heavy machine for evaluation.