This paper introduces a new family of prior models called Bernoulli-Gaussian-Mixtures (BGM), with a view to efficiently address sparse inverse problems in the Bayesian framework. The BGM family is based on continuous Location and Scale Mixtures of Gaussians (LSMG), which includes a wide range of symmetric and asymmetric heavy-tailed probability distributions. The decomposition of a distribution as a Gaussian mixture is a case of data augmentation from which we derive a Partially Collapsed Gibbs Sampler (PCGS) for the BGM, in a systematic way. the derived PCGS is shown to be more efficient than the standard Gibbs sampler, both in terms of number of iterations and CPU time. Moreover, special attention is paid to BGM involving a density defined over a real half-line. An asymptotically exact LSMG approximation is introduced, which allows us to expand the applicability of PCGS to cases such as BGM models with a non-negative support.