In this paper, we propose a new approach to the decentralized Simultaneous Localization And Mapping (SLAM) problem. The goal is to demonstrate the feasibility of decentralized localization using low-density maps built with low-cost sensors. This problem is challenging at different levels. Indeed, each vehicle localization tends to drift over time independently of one another making the global localization of a fleet hard to achieve. To counter this effect, called SLAM inconsistency and which has been stated numerous times in the literature, we introduce a model to represent the natural drift of SLAM algorithms. Its integration inside an Extended Kalman Filter is explained along with simulations validating its use. The second part of this paper presents the fusion architecture designed to solve the different problems arising in a decentralized scheme. It avoids data incest, which is an important source of inconsistency, and integrates the previously mentioned SLAM drift in the estimates produced. This architecture also separates the SLAM classically used for mono-vehicle applications from the high-level decentralized part offering flexibility regarding sensors and algorithms at a low-level. Other aspects, involved by the multi-vehicle settings, are also taken into account (communication losses, latencies, desynchronizations, unknown initial positions of the fleet members and data association). The whole algorithm has been tested in various scenarios with vehicles equipped with a single camera and an odometer. The results, from both simulated and real scenarios, show that our approach can work in real time with very small bandwidth requirements.