Enabling a rapid on-the-fly view of the content of a movie requires segmenting the movie and describing the segments in a user-compatible manner. The difficulty resides in extracting relevant semantic information from the audiovisual signal, both for the segmentation and the description. We introduce in this paper audio scenes and audio chapters in movies and present an algorithm for automatically segmenting a video based on the audio stream only. A tree-like audio-based structure of a video is proposed. Each scene at each abstraction level in the structure is classified into different scene categories. The automatic solution to audio scene and chapter segmentation and classification is evaluated on manually segmented and classified videos