We discuss the impact of asymmetries and noise on the nonlinear dynamics of vertical-cavity surface-emitting lasers (VCSEL). We focus in particular on the effects of these features on the chaotic dynamics that can be generated by a free-running VCSEL due to the intrinsic competition between polarization modes taking place in these devices. Experimentally, we observe significant asymmetries especially in the statistics of the chaotic dynamics. We show that these behaviour can be explained theoretically by a combined effect of the system asymmetries and the noise. This work therefore brings new light on the interplay between deterministic and stochastic processes taking place in VCSELs