In Tim Berners-Lee’s vision of the Semantic Web, software agents must be able to retrieve knowledge relevant to the end-user’s task, despite the heterogeneity and scale issues which are inherent to the web. Annotating web resources is considered a promising way to achieve this vision. We argue in this chapter that every actual use of a resource may provide useful knowledge about that resource and therefore should annotate it. We propose a framework for taking uses into account and annotating resources with their uses. This framework is the result of a generalization effort in our team after a number of projects focusing on modeling uses and experience. We believe that this framework is relevant to the Semantic Web where uses-based annotation may prove valuable.