In this paper we obtain two classification theorems for free boundary minimal hypersurfaces outside of the unit ball (exterior FBMH for short) in Euclidean space. The first result states that the only exterior stable FBMH with parallel embedded regular ends are the catenoidal hypersurfaces. To achieve this we prove a B\^ocher type result for positive Jacobi functions on regular minimal ends in $\mathbb{R}^{n+1}$ which, after some calculations, implies the first theorem. The second theorem states that any exterior FBMH $\Sigma$ with one regular end is a catenoidal hypersurface. Its proof is based on a symmetrization procedure similar to R. Schoen [14]. We also give a complete description of the catenoidal hypersurfaces, including the calculation of their indices.