Dispersing solid hard particles in an elasto-plastic material leads to important shear-history dependence of the behavior, namely strain hardening and Bauschinger effect. Strain hardening is observed as the progressive strengthening of a material during its plastic deformation and is usually associated with ductility, a property often sought after in composite materials to postpone fractures and failure. In addition, anisotropic mechanical properties are developed, the material resistance being larger in the direction of the imposed flow, which is referred to as the Bauschinger effect. We show that this is related here to shear-history-dependent particle-pair distribution functions. Roughness and interparticle contacts likely play a major role, as replacing hard particles by nondeformable bubbles modifies the suspension microstructure and suppresses strain hardening. Beyond suspensions, our study provides new insight in the understanding and control of strain hardening and Bauschinger effect in composite materials.