The presence of kiloparsec-sized bubble structures in both sides of the Galactic plan suggests active phases of Sgr A$^\star$, the central supermassive black hole of the Milky-Way in the last 1-6 Myr. The contribution of such event on the cosmic-ray flux measured in the solar neighborhood is investigated with numerical simulations. We evaluate whether the population of high-energy charged particles emitted by the Galactic Center could be sufficient to significantly impact the CR flux measured in the solar neighborhood. We present a set of 3D magnetohydrodynamical simulations, following the anisotropic propagation of CR in a Milky - Way like Galaxy. Independent populations of cosmic-ray are followed through time, originating from two different sources types, namely Supernovae and the Galactic Center. To assess the evolution of the CR flux spectrum properties, we split these populations into two independent energy groups of 100 GeV and 10 TeV. We find that the anisotropic nature of cosmic-ray diffusion dramatically affects the amount of cosmic-ray energy received in the solar neighborhood. Typical timescale to observe measurable changes in the CR spectrum slope is of the order 10 Myr, largely surpassing estimated ages of the Fermi bubbles in the AGN jet-driven scenario. We conclude that a cosmic-ray outburst from the Galactic center in the last few Myr is unlikely to produce any observable feature in the local CR spectrum in the TeV regime within times consistent with current estimates of the Fermi bubbles ages.