Action Text Diffusion Prior Network for Action Segmentation
Résumé
Action segmentation is a challenging task that requires accurate parsing and labeling of each action. There are two types of methods for action segmentation. The first type primarily focuses on extracting high-quality features from videos, while the second type focuses on combining textual and perceptual features through multimodal fusion. However, both types of methods have their limitations. The first type is limited to a single modality and does not leverage multimodal information, while the second type, although promising, is restricted by the language used to describe the actions in the texts. To solve these problems, we propose in this paper, an Action Text Diffusion Prior Network (ATDPN) which simultaneously improves the quality of the extracted visual features (by introducing a Video-level Diffusion Prior Sampling) and integrates the textual information to fullest extent. This leads to superior action segmentation results. Our experiments performed on GTEA dataset demonstrate the effective feature extraction ability of ATDPN.