The properties of vector vortex beams in vertical-cavity-surface emitting lasers with frequency-selective feedback is investigated. They are interpreted as high-order vortex solitons with a spatially non-uniform, but locally linear polarization state. In contrast to most schemes to obtain vector vortex beams relying on imprinting the polarization structure, vector vortex solitons form spontaneously due to the near polarization degeneracy in vertical-cavity devices. We observe radially, hyperbolic and spiral polarization configurations depending on small residual anisotropies in the system and multi-stability between different states. In addition, we demonstrate flip-flop operation of laser solitons via in principle local electronic nonlinearities. Combining the two themes might open up a route for a simple device enabling fast switching between different vector vortex beams for applications. The investigations connect nicely the fields of nonlinear science, singular optics, structured light and semiconductor laser technology.