Introduction: Control and optimization in queues have been an active area of research for decades, see for instance [10,11]. Most of the literature up to the present has focused on the model-based setting, a term used to describe the situation in which a model is known. In the coming years, we will witness a huge interest from the community in the model-free approach, a setting that does not assume knowledge of an exact underlying mathematical model. In this short note I provide a personal view of some of the challenges that lie ahead in the transition from model-based to model-free solutions in a queueing context.