An important issue for supporting multimedia applications in multiple heterogeneous networks, a typical pervasive computing environment, is how to optimize the rate allocation by intelligently utilizing the available network resources while, at the same time, to meet each application's QoS (Quality of Service) requirement. In this work, we develop and evaluate a rate allocation scheme in terms of audio and video applications based on a cross-layer design framework. At first, we construct a general distortion model according to the observed parameters in each network, as well as each application's rate-distortion characteristic. Then, the rate allocation is formulated as a convex optimization problem that minimizes the sum of the expected distortion of all applications. Furthermore, the realization of the distributed rate allocation algorithm for achieving an optimal or close-to-optimal end-to-end QoS under the overall limited resource budget is the highlight of this paper. Simulation results are provided which demonstrate the effectiveness of our proposed distributed rate allocation scheme.