In this paper, we consider an incompressible viscous fluid in an infinitely deep ocean, being bounded above by a free moving boundary. The governing equations are the gravity-driven incompressible Navier-Stokes equations with variable density and no surface tension is taken into account on the free surface. After using the Lagrangian transformation, we write the main equations in a perturbed form in a fixed domain. In the first part, we describe a spectral analysis of the linearized equations around a hydrostatic equilibrium $(\rho_0(x_3), 0, P_0(x_3)$ for a smooth increasing density profile $\rho_0$. Precisely, we prove that there exist infinitely many normal modes to the linearized equations by following the operator method initiated by Lafitte and Nguyễn. In the second part, we study the nonlinear Rayleigh-Taylor instability around the above profile by constructing a wide class of initial data for the nonlinear perturbation problem departing from the equilibrium, based on the finding of infinitely many normal modes. Our nonlinear result extends the previous framework of Guo and Strauss and also of Grenier.