Quantum field theories with quenched disorder are so hard to study that even exactly solvable free theories present puzzling aspects. We consider a free scalar field $\phi$ in $d$ dimensions coupled to a random source $h$ with quenched disorder. Despite the presence of a mass scale governing the disorder distribution, we derive a new description of the theory that allows us to show that the theory is gapless and invariant under conformal symmetry, which acts in a non-trivial way on $\phi$ and $h$. This manifest CFT description reveals the presence of exotic continuous symmetries, such as nilpotent bosonic ones, in the quenched theory. We also reconsider Cardy's CFT description defined through the replica trick. In this description, the nilpotent symmetries reveal a striking resemblance with Parisi-Sourlas supersymmetries. We provide explicit maps of correlation functions between such CFTs and the original quenched theory. The maps are non-trivial and show that conformal behaviour is manifest only when considering suitable linear combinations of averages of products of correlators. We also briefly discuss how familiar notions like normal ordering of composite operators and OPE can be generalized in the presence of the more complicated local observables in the quenched theory.