Energy harvesting is a promising approach to enable autonomous long-life wireless sensor networks. As typical energy sources present time-varying behavior, each node embeds an energy manager, which dynamically adapts the power consumption of the node to maximize the quality of service, while preventing power failure. In this work, RLMan, a novel energy management scheme based on reinforcement learning theory, is proposed. RLMan dynamically adapts its policy to time-varying environment by continuously exploring, while exploiting the current knowledge to improve the quality of service. The proposed energy management scheme has a very low memory footprint, and requires very few computational power, which makes it suitable for online execution on sensor nodes. Moreover, it only necessitates the state of charge of the energy storage device as an input, and therefore is practical to implement. RLMan was compared to three state-of-the-art energy management schemes, using simulations and energy traces from real measurements. Results show that using RLMan can enable almost 70 % gains regarding the average throughput.