This study aimed to determine whether horses have a kind of memory of humans (based on previous interactions), leading to a general significance of humans revealed by their reactions to humans in subsequent interactions. Subjects were 59 adult horses used to interact daily with humans. Three types of behavioural tests involving an unknown experimenter evaluated three possibly different memorized types of human-animal interactions (not work-related, using work-related objects, unfamiliar working task). We also performed standardized observations of routine interactions between each horse and its familiar handler (caretaker). To get a broad overview of the horses' reactions to humans, we recorded both investigative and aggressive behaviours during the tests, representing respectively a "positive" and a "negative" memory of the relationship. Whereas correlations between tests revealed a general perception of humans as either positive or negative, unusual tests, i.e. that are not usually performed, elicited more positive reactions. Moreover, some horses reacted positively to a motionless person in their box, but negatively when this same person approached them, for example for halter fitting. Overall, aggressive reactions were more reliable indicators of the relationship than positive reactions, both between tests and between familiar and unfamiliar humans. Our results also show generalization of the perception of humans. These results support our hypothesis that perception of humans by horses may be based on experience, i.e. repeated interactions. Altogether, our results support the hypothesis that horses can form a memory of humans that impacts their reactions in subsequent interactions.