Effective video-based Web information system deployment is still challenging, while the recent widespread of multimedia further raises the demand for new online audiovisual document edition and presentation alternatives. Hypervideo, a specialization of hypermedia focusing on video, can be used on the Web to provide a basis for video-centric documents and to allow more elaborated practices of online video. In this paper, we propose an annotation-driven model to conceptualize hypervideos, promoting a clear separation video content/metadata and their various potential presentations. Using the proposed model, features ofhypervideo are grafted to wider video-based Web documents in a Web standards-compliant manner. The annotation-driven hypervideo model and its implementation offer a general framework to experiment with new interaction modalities for video-based knowledge communication on the Web.