The handwriting recognition task is largely dominated by deep neural networks. However, it remains challenging for these advanced computer vision systems. Recently, the models have become more sophisticated, moving from line-level recognition to paragraph-level and even page-level recognition. In this paper, we will study those advances and the constraints that come with them, mainly focusing on two models we proposed: the Simple Predict & Align Network and the Vertical Attention Network. Both handle paragraph images, and we outperformed the state of the art on three datasets: RIMES, IAM and READ 2016