Cloud computing provides access to "infinite" storage and computing resources, offering promising perspectives for many applications (medicine, nuclear physics, meteorology, etc.). However, this new paradigm requires rethinking of database management principles in order to allow deployment on scalable and easy to access infrastructures, applying a pay-as-you-go model. This position paper introduces building blocks to provide cost-aware semantic caching. To this end, we first introduce cost models for data management in the cloud, then we present a semantic caching framework providing finely tuned caches for different data analysis systems. This semantic caching framework is then discussed in the context of our previous work on rewriting rules and cache management for OLAP queries. Finally, it discusses the problem of query evaluation in the cloud in presence of semantic caches as a multi-criteria optimization problem.