Q-routing is inspired by Q-learning, a reinforcement learning algorithm. Originally, it uses latency as routing metric. But, latency can be difficult to estimate especially in a noisy wireless environment. In this paper, we propose to filter the latency measure with a moving average, in order to improve the quality of service metrics such as packet delivery ratio and average delay. We compare our modification to the original Q-routing and use OLSRv2 as reference routing protocol. We observe an improvement of the average delivery time on a wireless grid of 3~\% compared to the original Q-routing. On our mobility scenario, the number of routes changes is at least twice lower (from 6 to 3 route changes between the two approaches in this scenario). The gain on QoS metrics depends mainly on the speed of the nodes. These improvements are obtained without making Q-routing more complex as a moving average is added. We provide all the materials to conduct reproducible research on our public git repository.