We propose an extension of the EM algorithm and its stochastic versions for the construction of incomplete data models when the selected model minimizes a penalized likelihood criterion. This optimization problem is particularly challenging in the context of incomplete data, even when the model is relatively simple. However, by completing the data, the E-step of the algorithm allows us to simplify this problem of complete model selection into a classical problem of complete model selection that does not pose any major difficulties. We then show that the criterion to be minimized decreases with each iteration of the algorithm. Examples of the use of these algorithms are presented for the identification of regression mixture models and the construction of nonlinear mixed-effects models.