The impacts of radiative, electrical and thermal losses on the performances of nanoscale-gap thermophotovoltaic (nano-TPV) power generators consisting of a gallium antimonide cell paired with a broadband tungsten and a radiatively-optimized Drude radiator are analyzed. Results reveal that surface mode mediated nano-TPV power generation with the Drude radiator outperforms the tungsten radiator, dominated by frustrated modes, only for a vacuum gap thickness of 10 nm and if both electrical and thermal losses are neglected. The key limiting factors for the Drude- and tungsten-based devices are respectively the recombination of electron-hole pairs at the cell surface and thermalization of radiation with energy larger than the cell absorption bandgap. A design guideline is also proposed where a high energy cutoff above which radiation has a net negative effect on nano-TPV power output due to thermal losses is determined. It is shown that the power output of a tungsten-based device increases by 6.5% while the cell temperature decreases by 30 K when applying a high energy cutoff at 1.45 eV. This work demonstrates that design and optimization of nano-TPV devices must account for radiative, electrical and thermal losses.