This paper addresses the stability and large-time behavior problem on the perturbations near the hydrostatic balance of the two dimensional Boussinesq system, taking into account vertical dissipation and horizontal thermal diffusion. The spatial framework $\Omega$ is defined as $ \mathbb{T}\times\mathbb{R}$, where $\mathbb{T}$ spans $[0, 1]$, representing the 1D periodic box, while $\mathbb{R}$ denotes the whole line. The results outlined in this article confirm the fact that the temperature can actually have a stabilizing effect on the buoyancy-driven fluids. The stability and long-time behavior issues discussed here are difficult due to the lack of the horizontal dissipation and vertical thermal diffusion. By formulating in the appropriate energy functional and implementing the orthogonal decomposition of the velocity and the temperature into their horizontal averages and oscillation parts, we are able to make up for the missing regularization and establish the nonlinear stability in the Sobolev space $H^2(\Omega)$ and acheive the algebraic decay rates for the oscillation parts in the $H^1$-norm.