We use light microscopy to investigate the aging dynamics of a glass made of closely packed soft spheres, following a rapid transition from a fluid to a solid-like state. By measuring time-resolved, coarse-grained displacements fields, we identify two classes of dynamical events, corresponding to reversible and irreversible rearrangements, respectively. The reversible events are due to the small, experimentally unavoidable fluctuations of the temperature imposed to the sample, leading to transient thermal expansions and contractions that cause shear deformations. The irreversible events are plastic rearrangements, induced by the repeated shear cycles. We show that the displacement due to the irreversible rearrangements grows linearly with time, both on average and at a local level. The velocity associated with this ballistic motion decreases exponentially with sample age, accounting for the observed slowing down of the dynamics. The displacement field due to the irreversible rearrangements has a vortex-like structure and is spatially correlated over surprisingly long distances.