This paper presents an efficient algebraic preconditioner to speed up the convergence of Fast Multipole accelerated Boundary Element Methods (FM-BEMs) in the context of time-harmonic 3D wave propagation problems and in particular the case of highly non-uniform discretizations. Such configurations are produced by a recently-developed anisotropic mesh adaptation procedure that is independent of partial differential equation and integral equation. The new preconditioning methodology exploits a complement between fast BEMs by using two nested GMRES algorithms and rapid matrix-vector calculations. The fast inner iterations are evaluated by a coarse hierarchical matrix (H-matrix) representation of the BEM system. These inner iterations produce a preconditioner for FM-BEM solvers. It drastically reduces the number of outer GMRES iterations. Numerical experiments demonstrate significant speedups over non-preconditioned solvers for complex geometries and meshes specifically adapted to capture anisotropic features of a solution, including discontinuities arising from corners and edges.