By a result known as Rieger's theorem (1956), there is a one-to-one correspondence, assigning to each cyclically ordered group $H$ a pair $(G,z)$ where $G$ is a totally ordered group and $z$ is an element in the center of $G$, generating a cofinal subgroup $\langle z\rangle$ of $G$, and such that the quotient group $G/\langle z\rangle$ is isomorphic to $H$. We first establish that, in this correspondence, the first order theory of the cyclically ordered group $H$ is uniquely determined by the first order theory of the pair $(G,z)$. Then we prove that the class of cyclically ordered groups is an elementary class and give an axiom system for it. Finally we show that, in opposition to the fact that all theories of totally Abelian ordered groups have the same universal part, there are uncountably many universal theories of Abelian cyclically ordered groups. We give for each of these universal theories an invariant, which is a pair of subgroups of the group of unimodular complex numbers.