In this paper, we propose a graph convolutional deep reinforcement learning framework for both smart load balancing and queuing agents in a collaborative environment. We aim to balance traffic loads on different paths, and then control how packets belonging to different flow classes are dequeued at network nodes. Our objective is twofold: first to improve general network performance in terms of throughput and end-to-end delay, and second, to ensure meeting stringent service level agreements for a set of classified network flows. Our proposals use attention mechanisms to extract relevant features from local observations and neighborhood policies to limit the overhead of inter-agent communications. We assess our algorithms in a Mininet testbed and show that they outperform classic approaches to load balancing and smart queuing in terms of throughput and end-to-end delay.