Pain assessment is a challenging problem in the field of emotion recognition. Pain represents a complex emotion difficult to detect or to estimate its intensity. This is what makes automatic pain assessment playing an important role in clinical diagnosis. Taking into consideration that pain generally generates spontaneous facial behaviour, these facial expressions could be used to detect the presence of pain. As a matter of fact, previous researches used machine learning and deep learning either to detect pain or to estimate pain level. In this paper, we propose a fine-tuning of pre-trained data-efficient image transformers and distillation (Deit) for pain detection from facial expressions. The effectiveness of the proposed architecture is evaluated on two publicly available databases, namely UNBC McMaster Shoulder Pain and BioVid Heat Pain. The proposed approach achieved promising preliminary results compared to the state of the art.