We propose new parametrizations for neural networks in order to estimate extreme Value-at-Risk and Expected-Shortfall in heavy-tailed settings. All proposed neural network estimators feature a bias correction based on an extension of the usual second-order condition to an arbitrary order. The convergence rate of the uniform error between extreme log quantities and their neural network approximation is established. The finite sample performances of the neural network estimator are compared to other bias-reduced extreme-value competitors on both real and simulated data. It is shown that our method outperforms them in difficult heavy-tailed situations where other estimators almost all fail.