Software-Defined Wide Area Network (SD-WAN) holds tremendous potential to provide multi-cloud multi-network interconnection and prevent channel congestion. However, traffic among Customer Premises Edge (CPE) and controllers continuously increases, requiring pre-emptive load balancing in the control plane. In this paper problem in SD-WANs when the controller presents a limited processing capacity. Specifically, the data plane may include one or more CPE deployed at a site where service traffic is forwarded. To address this narrow, we propose a new approach based on a Deep Reinforcement Learning (DRL) strategy to optimize the balancing process under a latency constraint. As far as we can tell, we have not observed any pertinent research published in this context. The obtained simulation results revealed that our proposed approach decreases the load balancing and outperforms other baseline methods.