Emerging service-oriented architectures are pushing towards on-demand and ''on the fly'' composition of applications and business processes. In order to support service composition, the underlying infrastructure must provide a facility for on-demand discovery of services and service components. Discovery becomes challenging when services span heterogenous and independently administrated domains. For inter-domain discovery to be achieved independently of domain-specific service discovery technologies, a middleware is needed to interface between the different discovery systems. In this paper, we present a novel open service discovery architecture (OSDA) designed to serve as an open, scalable and fault-tolerant middleware for cross-domain discovery. We demonstrate the implementation of OSDA using a set of mature technologies.