Currently, multimedia documents may have to be executed on multiple devices such as mobile phones, PDAs, desktop computers, set-top boxes, etc. Hence, usage and platform diversity requires document adaptation according to execution contexts, sometimes unpredictable at design time. We propose to abstract from format specific details by defining a structure which expresses a set of objects and the relations between them. In order to capture the spatio-temporal and hypermedia dimensions of the document, we point out in this paper that these objects are multimedia objects and hypermedia links, and that relations between them are spatio-temporal. In this context, adapting amounts to modify in a minimal way the document abstraction according to the target device constraints. In order to show the applicability of our framework, we implement a prototype which adapts SMIL documents.