In this work we explain how to properly use mean-field methods to solve the inverse Ising problem when the phase space is clustered, that is many states are present. The clustering of the phase space can occur for many reasons, e.g. when a system undergoes a phase transition. Mean-field methods for the inverse Ising problem are typically used without taking into account the eventual clustered structure of the input configurations and may led to very bad inference (for instance in the low temperature phase of the Curie-Weiss model). In the present work we explain how to modify mean-field approaches when the phase space is clustered and we illustrate the effectiveness of the new method on different clustered structures (low temperature phases of Curie-Weiss and Hopfield models).