We present a hidden Markov model-based approach to model on-line handwriting sequences. This problem is addressed in term of learning both hidden Markov models (HMM) structure and parameters from data. We iteratively simplify an initial HMM that consists in a mixture of as many left-right HMM as training sequences. There are two main applications of our approach: allograph identification and classification. We provide experimental results on these two different tasks.