Modeling the dynamics of physical systems recently gained attention in the machine learning community. Most recent works rely on complete observations of the physical state, whereas only partial observations are available in practice. Estimating the full state dynamics is important for the understanding of the underlying phenomenon and for model based prediction. Largely unexplored from an ML viewpoint, we address in this work the estimation and forecast of a partially observed spatio-temporal system, leveraging prior dynamical knowledge. To solve both forward (forecasting) and inverse (identification) problems, we bridge numerical models of partial differential equations and deep learning and introduce a dynamical regularization on the unobserved states. This constrains our estimation and improves estimation performances. The approach is validated on two simulated datasets where the dynamics is controlled and fully known.