Multiple light scattering is widely used to characterize dense colloidal systems as well as in deep tissue imaging; experiments are often interpreted via a theory of diffusion of the light intensity within a sample, neglecting the vector nature of the electromagnetic wave. Recent experiments on diffuse backscattering with linearly polarized light from colloidal suspensions of micron size particles were found to display strong intensity variations with fourfold rotational symmetry when observed through an arbitrarily oriented linear analyzer. We show that these polarization patterns are manifestations of a Berry phase of the multiple scattered beam.