In the theory of probabilistic model checking, recurrence theorems play crucial roles in reducing an infinitary question to finitary-and thus machine-checkable-ones. They do so specifically by (1) reducing recurrence specifications-which speak about infinite future-to finding bottom strongly connected components (BSCCs), much like the lasso-based algorithm for non-emptiness of Büchi automata, and (2) reducing (quantitative) computation of some probabilities to the (qualitative) graph-theoretic question of strong connectedness. In this paper, we present infinitary extensions of those recurrence theorems, ones that apply to Markov chains with infinite (and even continuous) state spaces. The extension calls for careful generalization of the original finitary arguments to topological ones. We conduct the extension, imposing Polishness and compactness as key conditions, and using upper semicontinuity as an important technical notion in our proofs.