Inspired by the pioneering work of H. Yilmaz, we propose in this paper a mathematical formalization of the relativistic nature of color perception that relies on a single axiom: the space of perceived colors is the positive cone of a three dimensional formally real Jordan algebra. If this Jordan algebra is taken to be the spin factor R ⊕ R^2 , then we get a natural description of a perceived color as a real positive magnitude, which encodes an information related to luminance, together with a chromatic vector, that carries an information related to hue and saturation. We show that chromatic vectors follow the relativistic addition rule, which turns out to be equivalent with the use of the Hilbert metric to measure chromatic differences. This metric is shown to be compatible with experimental data. Up to authors' knowledge, this work proofs that color perception is the first relativistic phenomenon not related to movement ever discovered in nature.