In this paper, we consider the eigenvalues and eigenvectors of finite, low rank perturbations of random matrices. Specifically, we prove almost sure convergence of the extreme eigenvalues and appropriate projections of the corresponding eigenvectors of the perturbed matrix for additive and multiplicative perturbation models. The limiting non-random value is shown to depend explicitly on the limiting spectral measure and the assumed perturbation model via integral transforms that correspond to very well known objects in free probability theory that linearize non-commutative free additive and multiplicative convolution. Moreover, we uncover a remarkable phase transition phenomenon whereby the large matrix limit of the extreme eigenvalues of the perturbed matrix differs from that of the original matrix if and only if the eigenvalues of the perturbing matrix are above a certain critical threshold. This critical threshold is intimately related to the same aforementioned integral transforms. We examine the consequence of this eigenvalue phase transition on the associated eigenvectors and generalize our results to examine the singular values and vectors of finite, low rank perturbations of rectangular random matrices. The analysis brings into sharp focus the analogous connection with rectangular free probability. Various extensions of our results are discussed.