In metal matrix composites made of elastic spherical reinforcements (zirconia and/or silica ceramics) embedded in an aluminium matrix two damage mechanisms are observed to trigger failure: particle fracture or decohesion at the particle-matrix interface. Experimentally, it was shown that the dominant damage mechanism is governed by the matrix plastic characteristics: a soft matrix composite shows interface debonding while particles breakdown occurs in the composite with a hard matrix. The physics underlying this dependence are related to plastic strain near the interface which can assist decohesion. We present a cohesive zone model for the particle-matrix interface that accounts for local plastic strain effects, based on the analysis of the influence of dislocation accumulation on the local stress level. Such a description is shown able to capture the experimental observations while being simple to implement in a finite element code, as performed here with Abaqus.