We consider the sub- or supercritical Neumann elliptic problem $-\Delta u+\mu u=u^{\frac{N+2}{N-2}+\epsilon}$, $u>0$ in $\Omega$; $\frac{\partial u}{\partial n}=0$ on $\partial\Omega$, $\Omega$ being a smooth bounded domain in $\mathbb{R}^{N}, N\geq 4, \mu>0$ and $\epsilon\neq0$ a small number. We show that for $\epsilon>0$, there always exists a solution to the slightly supercritical problem, which blows up at the most curved part of the boundary as $\epsilon$ goes to zero. On the other hand, for $\epsilon<0$, assuming that the domain is not convex, there also exists a solution to the slightly subcritical problem, which blows up at the least curved part of the domain.