In this paper, we show how the approach of nonsmooth dynamical systems can be used to develop a suitable method for the modelling of a rotary oil drilling system with friction. We study different kinds of frictions and analyse the mathematical properties of the involved dynamical systems. We show that using a general Stribeck model for the frictional contact, we can formulate the rotary drilling system as a well-posed evolution variational inequality. Several numerical simulations are also given to illustrate both the model and the theoretical results.