We present a new, publicly available library of dust spectral energy distributions (SEDs). These SEDs are characterized by only three parameters: the dust mass (Mdust), the dust temperature (Tdust), and the mid-to-total infrared color (IR8 ≡ LIR/L8). The latter measures the relative contribution of polycyclic aromatic hydrocarbon (PAH) molecules to the total infrared luminosity. We used this library to model star-forming galaxies at 0.5 < z < 4 in the deep CANDELS fields, using both individual detections and stacks of Herschel and ALMA imaging, and extending this sample to z = 0 using the Herschel Reference Survey. At first order, the dust SED of a galaxy was observed to be independent of stellar mass, but evolving with redshift. We found trends of increasing Tdust and IR8 with redshift and distance from the SFR–M∗ main sequence, and quantified for the first time their intrinsic scatter. Half of the observed variations of these parameters was captured by the above empirical relations, and after subtracting the measurement errors we found residual scatters of ΔTdust/Tdust = 12% and Δlog IR8 = 0.18 dex. We observed second order variations with stellar mass: massive galaxies (M∗ > 1011M⊙) at z ≤ 1 have slightly lower temperatures indicative of a reduced star formation efficiency, while low mass galaxies (M∗ < 1010M⊙) at z ≥ 1 showed reduced PAH emission, possibly linked to their lower metallicities. Building on these results, we constructed high-fidelity mock galaxy catalogs to predict the accuracy of infrared luminosities and dust masses determined using a single broadband measurement. Using a single James Webb Space Telescope (JWST) MIRI band, we found that LIR is typically uncertain by 0.15 dex, with a maximum of 0.25 dex when probing the rest-frame 8 μm, and this is not significantly impacted by typical redshift uncertainties. On the other hand, we found that ALMA bands 8 to 7 and 6 to 3 measured the dust mass at better than 0.2 and 0.15 dex, respectively, and independently of redshift, while bands 9 to 6 only measured LIR at better than 0.2 dex at z > 1, 3.2, 3.8, and 5.7, respectively. Starburst galaxies had their LIR significantly underestimated when measured by a single JWST or ALMA band, while their dust mass from a single ALMA band were moderately overestimated. This dust library and the results of this paper can be used immediately to improve the design of observing proposals, and interpret more accurately the large amount of archival data from Spitzer, Herschel and ALMA.