Yield stress fluids (YSFs) display a dual nature highlighted by the existence of a critical stress σy such that YSFs are solid for stresses σ imposed below σy, whereas they flow like liquids for σ>σy. Under an applied shear rate ˙γ , the solid-to-liquid transition is associated with a complex spatiotemporal scenario that depends on the microscopic details of the system, on the boundary conditions, and on the system size. Still, the general phenomenology reported in the literature boils down to a simple sequence that can be divided into a short-time response characterized by the so-called “stress overshoot,” followed by stress relaxation towards a steady state. Such relaxation can be either (1) long-lasting, which usually involves the growth of a shear band that can be only transient or that may persist at steady state or (2) abrupt, in which case the solid-to-liquid transition resembles the failure of a brittle material, involving avalanches. In the present paper, we use a continuum model based on a spatially resolved fluidity approach to rationalize the complete scenario associated with the shear-induced yielding of YSFs. A key feature of our model is to provide a scaling for the coordinates of the stress overshoot, i.e., stress σM and strain γM as a function of ˙γ , which shows good agreement with experimental and numerical data extracted from the literature. Moreover, our approach shows that the power-law scaling σM( ˙γ ) is intimately linked to the growth dynamics of a fluidized boundary layer in the vicinity of the moving boundary. Yet such scaling is independent of the fate of that layer, and of the long-term behavior of the YSF, i.e., whether the steady-state flow profile is homogeneous or shear-banded. Finally, when including the presence of “long-range” correlations, we show that our model displays a ductile to brittle transition, i.e., the stress overshoot reduces into a sharp stress drop associated with avalanches, which impacts the scaling σM( ˙γ ). This generalized model nicely captures subtle avalanche-like features of the transient shear banding dynamics reported in experiments. Our work offers a unified picture of shear-induced yielding in YSFs, whose complex spatiotemporal dynamics are deeply connected to nonlocal effects.