In a single-pixel camera, an unknown object is sequentially illuminated by intensity patterns. The total reflected or transmitted intensity is summed in a single-pixel detector from which the object is computationally reconstructed. In the situation where the measurements are limited by photon-noise, it is questionable whether a single-pixel camera performs better or worse than simply scanning the object with a focused intensity spot— a modality known as point raster scanning and employed in many laser scanning systems. Here, we solve this general question and report that positive intensity modulation based on Hadamard or Cosine patterns does not necessarily improve the single-to-noise ratio (SNR) of single-pixel cameras, as compared to point raster scanning (RS). Instead, we show that the SNR is only improved on object pixels at least k times brighter than the object mean signal, where k is a constant that depends on the modulation scheme (modulation matrix, number of detectors, etc). The constant k is derived for several widespread cases and has important consequences on the choice of the optical deign. This fundamental property is demonstrated theoretically, numerically, and is experimentally confirmed in the spatial domain (wide-field fluorescence imaging) and in the spectral domain (spontaneous Raman spectral measurements). Finally, we provide user-oriented guidelines that help decide when and how multiplexing under photon-noise should be used instead of point RS.