Consider a real diagonal deterministic matrix $X_n$ of size $n$ with spectral measure converging to a compactly supported probability measure. We perturb this matrix by adding a random finite rank matrix of a certain form, with delocalized eigenvectors. We show that the joint law of the extreme eigenvalues of the perturbed model satisfies a large deviation principle, in the scale $n$ with a good rate function for which we give a variational expression. We tackle both cases when the extreme eigenvalues of $X_n$ converge to the edges of the support of the limiting measure and when we allow some eigenvalues of $X_n$, that we call outliers, to converge out of the bulk. We can also generalize our results to the case when $X_n$ is random, with law proportional to $e^{- Trace V(X)} dX$ for $V$ growing fast enough at infinity and any perturbation of finite rank with orthonormal eigenvectors.