When processing audio data, multiple challenges arise, one of them being the diversity of information present in the audio signal. Various audio segmentation subtasks appeared includ- ing voice activity detection (VAD), overlapped speech detection (OSD), music or noise detection. These tasks are often com- pleted by separate models trained on different datasets, thus in- creasing computational costs and limiting the usage to specific datasets. We first show that a multiclass VAD and OSD model outperforms state of the art models. Then, we propose 3MAS, a novel deep learning-based audio segmentation model capable of handling multiple datasets, and assessing multiple simulta- neously as a multilabel segmentation problem. 3MAS provides similar performances as specialized models with a similar archi- tecture and can be trained using partial and unbalanced annota- tions on different datasets. 3MAS is a gain in computational time, and opens new opportunities to include new labels.