We investigate a bi-layer scheme for circularly polarized infrared thermal radiation. Our approach takes advantage of the strong anisotropy of low-symmetry materials such as β-Ga2O$_3$ and α-MoO$_3$. We numerically report narrow-band, high degree of circular polarization (over 0.85), thermal radiation at two typical emission frequencies related to the excitation of β-Ga$_2$O$_3$ optical phonons. Optimization of the degree of circular polarization is achieved by a proper relative tilt of the crystal axes between the two layers. Our simple but effective scheme could set the basis for a new class of lithography-free thermal sources for IR bio-sensing.