Time-of-flight secondary ion mass spectrometry (ToF-SIMS) is increasingly used to analyze cultural heritage materials because it can simultaneously detect organic and inorganic materials while mapping them on a surface. The precise identification of a pigment in a specific layer of a painting or of remaining color on a statue can inform about the technique used or the time of manufacture as well as expose possible forgeries when anachronistic ingredients are identified. Reference spectra are required to confidently identify a given pigment using ToF-SIMS. This paper focuses on eight pigments containing copper, zinc, arsenic, or phosphate, all manufactured following historical recipes. The negative polarity ToF-SIMS reference spectra using a Bi3+ primary ion species are presented here. Presented together, these spectra and corresponding tables of secondary ions provide a valuable help in differentiating these pigments because copper, zinc, arsenic, or phosphate, combined with oxygen, share many mass interferences.