A generalized Swendsen-Wang (GSW) algorithm is proposed for the joint segmentation of a set of multiple images sharing, in part, an unknown number of common classes. The class labels are a priori mod-eled by a combination of the hierarchical Dirichlet process (HDP) and the Potts model. The HDP allows the number of regions in each image and classes to be automatically inferred while the Potts model ensures spatially consistent segmentations. Compared to a classical Gibbs sampler, the GSW ensures a better exploration of the posterior distribution of the labels. To avoid label switching issues, the best partition is estimated using the Dahl's criterion.