In this paper we consider conditionally Gaussian state space models with Markovian switches and we propose a new method of approximating the optimal solution by the use of Markov chains hidden with long memory noise. We show through experiments that our method can be more efficient than the classical particle filter based approximation. Keywords: Conditionally Gaussian state space model, Markov switching, Markov chains hidden with long memory noise, Expectation-Maximization, Iterative conditional estimation