Quadratic backward stochastic differential equations with singularity in the value process appear in several applications, including stochastic control and physics. In this paper, we prove existence and uniqueness of equations with generators (dominated by a function) of the form $|z|^2/y$. In the particular case where the BSDE is Markovian, we obtain existence of viscosity solutions of singular quadratic PDEs with and without Neumann lateral boundaries, and rather weak assumptions on the regularity of the coefficients. Furthermore, we show how our results can be applied to some optimization problems in finance.