Mixed-State conditionally heteroscedastic latent factor models attempt to describe a complex nonlinear dynamic system with a succession of linear latent factor models indexed by a switching variable. Unfortunately, despite the framework’s simplicity exact state and parameter estimation are still intractable because of the interdependency across the latent factor volatility processes. Recently, a broad class of learning and inference algorithms for time series models have been successfully cast in the framework of dynamic Bayesian networks (DBN). This paper describes a novel DBN-based switching conditionally heteroscedastic latent factor model. The key methodological contribution of this paper is the novel use of the Generalized Pseudo-Bayesian method GPB2, a structured variational learning approach and an approximated version of the Viterbi algorithm in conjunction with the EM algorithm for overcoming the intractability of exact inference in mixed-state latent factor model. The conditional EM algorithm that we have developed for the maximum likelihood estimation, is based on an extended switching Kalman filter approach which yields inferences about the unobservable path of the common factors and their variances, and the latent variable of the state process. Extensive Monte Carlo simulations show promising results for tracking, interpolation, synthesis, and classification using learned models.