For some problems, we generally know the structure of the problem (i.e., the objective function to optimize and the constraints to respect), but we may be missing some contextual elements (e.g., some parameters of the model) since they may be too difficult or impossible to collect. Furthermore, users of the optimization system generally have tacit knowledge due to their experience. They are, thus, generally able to adapt the proposed solution in accordance with this tacit knowledge. They may, however, do this in a sub-optimal way. In terms of the Traveling Salesman Problem with Time Windows (TSP-TW), this implicit knowledge can be measured in how different are the routes that companies propose to drivers and the routes that these drivers do. We implement two Reinforcement Learning techniques that will learn from driver’s routes and will incorporate this tacit knowledge into a TSP-TW model. With the combination of Machine Learning and Operation Research techniques, we will be able to offer a procedure that adapts over time to the driver’s knowledge while minimizing route duration. To illustrate how the algorithms can learn over time, we will show a results comparison with some baseline benchmarks over instances with up to 100 customers.