This article presents the design and implementation of a fault tolerant architecture for sensor fusion that tolerates faults on a quadrotor unmanned aerial vehicle (UAV). It aims to tolerate both hardware sensors faults (GPS jamming, IMU lock or freezing, magnetometer sensitivity to high power magnetic fields...) and software faults (faults in the Kalman filter, bad parameters initialization....). The proposed architecture uses data fusion with Kalman filters in order to estimate the states (position and orientation) of the UAV. It includes an analytical redundancy using the dynamic model of the system. The estimations of the defined Kalman filters and the dynamic model feed a weighted average voter, which increases the accuracy of the outputs and the error detection process. The proposed architecture allows multiple recovery solutions to a faulty system and thus increasing its flexibility. The architecture is validated using numerical simulations and experimental flights in real outdoor environment using a quadrotor.