This paper investigates the dynamic behavior of a Van der Pol oscillator (used as an archetypal self-sustained oscillator) coupled to a bistable nonlinear energy sink (BNES). We first show using numerical simulations that this system can undergo a multitude of motions including different types of periodic regimes and so-called strongly modulated responses (SMR) as well as chaotic regimes. We also show that a BNES can be much more efficient than a classical cubic NES but this is not robust since a little perturbation can switch the system from harmless to harmful situations. However, even in the most unfavorable cases, it is possible to find a set of parameters for which the BNES performs better than the NES. A multiple time scales approach is then addressed to analyze the system. In this context, we show that the so-called Multiple Scale/Harmonic Balance Method (MSHBM) must be modified (compared to its usual use) to consider the specific feature of the BNES, i.e., that it can have a nonzero-mean oscillating motion. This allows us to derive a so-called amplitude-phase modulation dynamics (APMD) which can reproduce the complex behavior of the initial system. Because of the presence of a small perturbation parameter (i.e., the mass ratio between the BNES and the VdP oscillator), the APMD is governed by two different time scales. More precisely, it appears as a (3,1)fast-slow system whose motion is constituted in a succession of slow and fast epochs. Founding a (3,1)-fast-slow APMD is interesting since that implies a more complex dynamics than in the case of a classic NES whose APMD is only (2,1)-fast-slow. A fast-slow analysis is finally conducted within the framework of the geometric singular perturbation theory. From the computation of the so-called critical manifold and the analytical expressions of the APMD fixed points, a global stability analysis is performed. This enables us to interpret a certain number of regimes observed on numerical simulations of the initial system.