In this article, we study the excursions sets $\mathcal{D}_p=f^{-1}([-p,+\infty[)$ where $f$ is a natural real-analytic planar Gaussian field called the Bargmann-Fock field. More precisely, $f$ is the centered Gaussian field on $\mathbb{R}^2$ with covariance $(x,y) \mapsto \exp(-\frac{1}{2}|x-y|^2)$. In [BG16], Beffara and Gayet prove that, if $p \leq 0$, then a.s. $\mathcal{D}_p$ has no unbounded component. We show that conversely, if $p>0$, then a.s. $\mathcal{D}_p$ has a unique unbounded component. As a result, the critical level of this percolation model is $0$. We also prove exponential decay of crossing probabilities under the critical level. To show these results, we develop several tools including a KKL-type result for biased Gaussian vectors (based on the analogous result for product Gaussian vectors by Keller, Mossel and Sen in [KMS12]) and a sprinkling inspired discretization procedure. These intermediate results hold for more general Gaussian fields, for which we prove a discrete version of our main result.