A remarkable feature of active matter is the propensity to self-organize. One striking instance of this ability to generate spatial structures is the cluster phase, where clusters broadly distributed in size constantly move and evolve through particle exchange, breaking or merging. Here we propose an exhaustive description of the cluster dynamics in apolar active matter. Exploiting large statistics gathered on thousands of Janus colloids, we measure the aggregation and fragmentation rates and rationalize the resulting cluster size distribution and fluctuations. We also show that the motion of individual clusters is entirely consistent with a model positing random orientation of colloids. Our findings establish a simple, generic model of cluster phase, and pave the way for a thorough understanding of clustering in active matter.