Background Bacterial communities that are associated with tropical reef-forming corals are being increasingly recognized for their role in host physiology and health. However, little is known about the microbial diversity of the communities associated with temperate gorgonian corals, even though these communities are key structural components of the ecosystem. In the Northwestern Mediterranean Sea, gorgonians undergo recurrent mass mortalities, but the potential relationship between these events and the structure of the associated bacterial communities remains unexplored. Because microbial assemblages may contribute to the overall health and disease resistance of their host, a detailed baseline of the associated bacterial diversity is required to better understand the functioning of the gorgonian holobiont. Methodology/Principal Findings The bacterial diversity associated with the gorgonian Paramuricea clavata was determined using denaturing gradient gel electrophoresis, terminal-restriction fragment length polymorphism and the construction of clone libraries of the bacterial 16S ribosomal DNA. Three study sites were monitored for 4 years to assess the variability of communities associated with healthy colonies. Bacterial assemblages were highly dominated by one Hahellaceae-related ribotype and exhibited low diversity. While this pattern was mostly conserved through space and time, in summer 2007, a deep shift in microbiota structure toward increased bacterial diversity and the transient disappearance of Hahellaceae was observed. Conclusion/Significance This is the first spatiotemporal study to investigate the bacterial diversity associated with a temperate shallow gorgonian. Our data revealed an established relationship between P. clavata and a specific bacterial group within the Oceanospirillales. These results suggest a potential symbiotic role of Hahellaceae in the host-microbe association, as recently suggested for tropical corals. However, a transient imbalance in bacterial associations can be tolerated by the holobiont without apparent symptoms of disease. The subsequent restoration of the Hahellaceae-dominated community is indicative of the specificity and resilience of the bacteria associated with the gorgonian host.