In this paper, an evidential fusion scheme between LIDAR scans and RGB images is proposed. To assess the drivability of an area met by an autonomous vehicle, LIDAR points are classified as either belonging to the ground, or not, and the RGB image is processed by a state-of-the-art convo-lutional neural network to obtain semantic labels. The results are then fused into an evidential grid, to handle incoherences over time and between sensors. The dynamic behaviour of potentially moving objects can be estimated from the high-level semantic labels obtained from the semantic segmentation of the image. LIDAR scans and images are not assumed to be acquired at the same time, making the proposed grid mapping algorithm asynchronous. This approach is justified by the need for handling, at the same time, sensor uncertainties, incoherences of results over time and between sensors, and the need for handling sensor failure. In classical LIDAR/camera fusion, in which LIDAR scans and images are considered to be acquired at the same time (or synchronously), the failure of a single sensor leads to the failure of the whole fusion system. However, experiments on a challenging use case show how this approach can be used to fuse contradictory information over time, while allowing the vehicle to operate even in case of the failure of a single sensor.