In this paper, we formulate the traffic-aware mobile nodes sleeping with traffic offloading as a Markov Decision Process (MDP) and solve it using Deep Reinforcement Learning (DRL). Our model characterizes jointly the energy saving actions due to base stations entering in sleep mode as well offloading options to neighboring nodes of the turned off gNodeB. To solve this problem, the Proximal Policy Optimization (PPO) integrated with action masking is leveraged. Our validation results, when training the model with open source datasets, show a potential of reducing up to 16% of the network energy consumption without negatively affecting traffic coverage.