The mechanical non-linear response of dense Brownian suspensions of polymer gel particles is studied experimentally and by means of numerical simulations. It is shown that the response to the application of a constant shear rate depends on the previous history of the suspension. When the flow starts from a suspension at rest, it exhibits an elastic response followed by a stress overshoot and then a plastic flow regime. Conversely, after flow reversal, the stress overshoot does not occur, and the apparent elastic modulus is reduced while numerical simulations reveal that the anisotropy of the local microstructure is delayed relative to the macroscopic stress.