Across many languages, pronouns are the most frequently produced referring expressions. We examined whether and how speakers avoid referential ambiguity that arises when the gender of a pronoun is compatible with more than one entity in the context in French. Experiment 1 showed that speakers use fewer pronouns when human referents have the same gender than when they had different genders, but grammatical gender congruence between inanimate referents did not result in fewer pronouns. Experiment 2 showed that semantic similarity between non-human referents can enhance the likelihood that speakers avoid grammatical-gender ambiguous pronouns. Experiment 3 pitched grammatical gender ambiguity avoidance against the referents' competition in the non-linguistic context, showing that when speakers can base their pronoun choice on non-linguistic competition, they ignore the pronoun's grammatical gender ambiguity even when the referents are semantically related. The results thus indicated that speakers preferentially produce referring expressions based on non-linguistic information; they are more likely to be affected by the referents' non-linguistic similarity than by the linguistic ambiguity of a pronoun.