The invariance principle for M/M/1 and M/M/∞ queues states that when properly renormalized (i.e. rescaled and centered), the Markov processes which describe these systems both converge to a diffusive limit when the driving parameters go to infinity: a killed Brownian motion in the former case and an Ornstein-Uhlenbeck process for the latter. The purpose of this paper is to assess the rate of convergence in these diffusion approximations. To this end, we extend to these contexts, the functional Stein's method introduced for the Brownian approximation of Poisson processes.