We study the Lie algebra of infinitesimal isometries on compact Sasakian and K-contact manifolds. On a Sasakian manifold which is not a space form or 3-Sasakian, every Killing vector field is an infinitesimal automorphism of the Sasakian structure. For a manifold with K-contact structure, we prove that there exists a Killing vector field of constant length which is not an infinitesimal automorphism of the structure if and only if the manifold is obtained from the Konishi bundle of a compact pseudo-Riemannian quaternion-Kähler manifold after changing the sign of the metric on a maximal negative distribution. We also prove that nonregular Sasakian manifolds are not homogeneous and construct examples with cohomogeneity one. Using these results we obtain in the last section the classification of all homogeneous Sasakian manifolds.