In this paper we study the quantum learnability of constant-depth classical circuits under the uniform distribution and in the distribution-independent framework of PAC learning. In order to attain our results, we establish connections between quantum learning and quantum-secure cryptosystems. We then achieve the following results. 1) Hardness of learning AC$^0$ and TC$^0$ under the uniform distribution. Our first result concerns the concept class TC$^0$ (resp. AC$^0$), the class of constant-depth and polynomial-sized circuits with unbounded fan-in majority gates (resp. AND, OR, NOT gates). We show that if there exists no quantum polynomial-time (resp. strong sub-exponential time) algorithm to solve the Ring Learning with Errors (RLWE) problem, then there exists no polynomial-time quantum learning algorithm for TC$^0$ (resp. AC$^0$) under the uniform distribution (even with access to quantum membership queries). The main technique in this result uses explicit pseudo-random functions that are believed to be quantum-secure to construct concept classes that are hard to learn quantumly under the uniform distribution. 2) Hardness of learning TC$^0_2$ in the PAC setting. Our second result shows that if there exists no quantum polynomial time algorithm for the LWE problem, then there exists no polynomial time quantum PAC learning algorithm for the class TC$^0_2$, i.e., depth-2 TC$^0$ circuits. The main technique in this result is to establish a connection between the quantum security of public-key cryptosystems and the learnability of a concept class that consists of decryption functions of the cryptosystem. This gives a strong (conditional) negative answer to one of the "Ten Semi-Grand Challenges for Quantum Computing Theory" raised by Aaronson [Aar05], who asked if AC$^0$ and TC$^0$ can be PAC-learned in quantum polynomial time.