People are increasingly demanding rich-media and bundled services. However, the diverse terminals, heterogeneous networks as well as various user requirements constrain the multimedia access to low quality in the pervasive computing environment. In order to enable rich-media delivery across a wide range of devices and networks, multimedia adaptation with scalable QoS management is an important issue. Our research introduces a Scalable Multimedia Delivery (SMD) framework with QoS management. This framework utilizes the CAM4Home metadata model to aggregate multimodal rich media services into a bundle. MPEG-21 metadata is integrated into the CAM4Home model to enforce interoperable QoS management. The issues in supporting QoS are addressed on both fidelity and modality. We further develop the SMD system in IP Multimedia Subsystem (IMS) architecture, where multimedia adaptation is implemented through application-level QoS negotiation.