Given an infinite word with values in a finite alphabet that admit frequencies for all its factors, the smallest frequency function associates with a given positive integer the smallest frequency for factors of this length, and the recurrence function measures the gaps between successive occurrences of factors. We develop in this paper the relations between the growth orders of the smallest frequency and of the recurrence functions. As proved by M. Boshernitzan, linearly recurrent words are characterized in terms of the smallest frequency function. In this paper, we see how to extend this result beyond the case of linearly recurrent words. We first establish a general relation between the smallest frequency and the recurrence fuctions. Given a lower bound for the smallest frequency function, this relation provides an upper bound for the recurrence function which involves the primitive of this lower bound. We then see how to improve this relation in the case of Sturmian words where the product of the smallest frequency and the recurrence function is bounded for most of the lengths of factors. We also indicate how to construct Sturmian words having a prescribed behaviour for the smallest frequency function, and for the product of the smallest frequency and the recurrence function.