We consider the supercritical Dirichlet problem $$\left(P_\epsilon\right)\qquad -\Delta u=u^{{N+2\over N-2}+\epsilon}\ \hbox{in $\Omega$},\ u>0\ \hbox{in $\Omega,$}\ u=0\ \hbox{on $\partial\Omega$} $$ where $N\ge3,$ $\epsilon>0$ and $\Omega\subset\mathbb{R}^N$ is a smooth bounded domain with a small hole of radius $d.$ When $\Omega$ has some symmetries, we show that $\left(P_\epsilon\right)$ has an arbitrary number of solutions for $\epsilon$ and $d$ small enough. When $\Omega$ has no symmetries, we prove the existence, for $d$ small enough, of solutions blowing up at two or three points close to the hole as $\epsilon$ goes to zero.