Due to contextual traffic conditions, the computation of optimized or shortest paths is a very complex problem for both drivers and autonomous vehicles. This paper introduces a reinforcement learning mechanism that is able to efficiently evaluate path durations based on an abstraction of the available traffic information. The authors demonstrate that a cache data structure allows a permanent access to the results whereas a lazy politics taking new data into account is used to increase the viability of those results. As a client of the proposed learning system, the authors consider a contextual path planning application and they show in addition the benefit of integrating a client cache at this level. Our measures highlight the performance of each mechanism, according to different learning and caching strategies.