When studying quantum field theories and lattice models, it is often useful to analytically continue the number of field or spin components from an integer to a real number. In spite of this, the precise meaning of such analytic continuations has never been fully clarified, and in particular the symmetry of these theories is obscure. We clarify these issues using Deligne categories and their associated Brauer algebras, and show that these provide logically satisfactory answers to these questions. Simple objects of the Deligne category generalize the notion of an irreducible representations, avoiding the need for such mathematically nonsensical notions as vector spaces of non-integer dimension. We develop a systematic theory of categorical symmetries, applying it in both perturbative and non- perturbative contexts. A partial list of our results is: categorical symmetries are preserved under RG flows; continuous categorical symmetries come equipped with conserved currents; CFTs with categorical symmetries are necessarily non-unitary.