Efficient services composition for grid-enabled data-intensive applications
Abstract
The problem we address in this paper is to build complex applications by reusing and assembling scientific codes on a production grid infrastructure. We first review the two main paradigms for executing application code on a grid: (a) the task based approach, associated to global computing, characterized by its efficiency, and (b) the service approach developed in the meta computing and the Internet communities, characterized by its flexibility. We carefully introduce the terminologies coming from different computing communities and we combine the advantages of both approaches. We argue that the service approach is more relevant for composing application codes. However, (i) it introduces an extra level of complexity on the application developer side and (ii) it makes application optimization on a grid more difficult. We propose solutions based on (i) a generic code wrapper which simplifies new application services development and (ii) a new grouping strategy coupled with traditional data and services parallelization techniques, that proved to be underused in the grid area for optimization. These solutions are implemented in our grid workflow enactor prototype. To our knowledge, it is the first implementation taking into account all these optimization levels. The theoretical performances are analyzed and compared to real measurements. Results are shown on a real, data-intensive application to medical images analysis. The execution is done on the very large scale EGEE production infrastructure. An execution time speed up of approximately 9 is achieved, with a minimal additional cost for paplication developers. The strategies adopted being generic, many other applications could similarly be instrumented at very low cost. Moreover, our optimized application enactor is based on existing standards and it is freely available for download.
Origin : Files produced by the author(s)
Loading...