We study the approximation of the spectrum of a second-order elliptic differential operator by the Hybrid High-Order (HHO) method. The HHO method is formulated using cell and face unknowns which are polynomials of some degree $k\geq0$. The key idea for the discrete eigenvalue problem is to introduce a discrete operator where the face unknowns have been eliminated. Using the abstract theory of spectral approximation of compact operators in Hilbert spaces, we prove that the eigenvalues converge as $h^{2t}$ and the eigenfunctions as $h^{t}$ in the $H^1$-seminorm, where $h$ is the mesh-size, $t\in [s,k+1]$ depends on the smoothness of the eigenfunctions, and $s>\frac12$ results from the elliptic regularity theory. The convergence rates for smooth eigenfunctions are thus $h^{2k+2}$ for the eigenvalues and $h^{k+1}$ for the eigenfunctions. Our theoretical findings, which improve recent error estimates for Hybridizable Discontinuous Galerkin (HDG) methods, are verified on various numerical examples including smooth and non-smooth eigenfunctions. Moreover, we observe numerically in one dimension for smooth eigenfunctions that the eigenvalues superconverge as $h^{2k+4}$}for a specific value of the stabilization parameter.