This paper describes the LORIA first participation at the SPMRL Shared Task. The focus of this work is on exploring several options to take advantage of the unlabeled data to improve the performances of a baseline dependency parser, which has neither be tuned to the specificities of the shared task nor evaluation languages. The semi-supervised approaches investigated include LDA word classes and super-tags predicted by a linear classifier trained with self-training. None of these options resulted in increased parsing accuracy.