We describe an hybrid model that combines deep neural networks (DNN) for nonlinear feature extraction and hidden conditional random fields (HCRF), i.e. conditional random fields with hidden states. The model is globally trained though joint optimization of HCRF and DNN parameters. To deal with this highly non convex optimization criterion, we propose a multi-step training which aims at providing a good initialization before the final joint optimization of all parameters. We investigate then the discriminative power of these models with respect to the architecture of the DNN, and compare our models to HMM and HCRF based algorithms on the IAM database.