Context-aware computing applications dynamically change their behaviors based on the context data and services provided by multiple physical spaces in the real world, such as persons, shops, homes and offices. In this paper, we propose a generic context gateway framework to manage these heterogeneous physical spaces of context sources. The framework forms a key enabling component of a scalable context-aware middleware that enables the effective access of context data and services from physical spaces. We have designed and implemented a number of gateway functions in our framework to facilitate the context data acquisition, processing and communication. These include diversified services with plug-in management, a query processing engine plus context reasoning capability, as well as the global and local connection between multiple gateways. The functionality and technical feasibility of the gateway have been demonstrated in our system prototype.