We extend simple opinion models to obtain stable but continuously evolving communities. Our scope is to meet a challenge raised by sociologists of generating ''structures that last from non lasting entities''. We achieve this by introducing two kinds of noise on a standard opinion model. First, agents may interact with other agents even if their opinion difference is large. Second, agents randomly change their opinion at a constant rate. We show that for a large range of control parameters, our model yields stable and fluctuating polarized states, where the composition and mean opinion of the emerging groups is fluctuating over time.