Learning to rank is an important problem in many sectors ranging from social sciences to artificial intelligence. However, it remains a rather difficult task to perform. Therefore, in some cases, it is preferable to perform cautious inference. For this purpose, we look into the possibility of an imprecise probabilistic approach for the Plackett-Luce model, a popular probabilistic model for label ranking. We aim at extending current Bayesian inference techniques for the Plackett-Luce model to an imprecise probabilistic setting so that we can deal with heterogeneous data by means of cautious mixture modelling. To achieve this, we perform a robust Bayesian analysis over a set of imprecise Dirichlet priors, which allows us to perform cautious label ranking. Finally, we use a synthetic dataset to illustrate our imprecise estimation method.