Pedestrian behavior is based both on cognitive processes and on social knowledge and representations. The presentation of the results of various studies we conducted will address the role of the physical environment but also of the perception of the rules and their learning on pedestrian decision-making. The aim is to show how knowledge of social norms can be a challenge for the autonomous vehicle. Our observations on pedestrian behaviors and decision-making show that behaviors in the road space mobilize prescriptive and descriptive norms and that pedestrian compliance with legal rules varies according to context, the crossing facilities, the type of built environment, the presence or absence of traffic and other pedestrians and whether or not the pedestrian mode is constrained. In particular, this contextual compliance varies according to the inferences pedestrians build concerning the pedestrian-driver balance of power, based on some clues found in the environment, in particular the type and function of the built environment, the width of the sidewalks and the presence parking spaces. Compliance with road rules is not only subject to the local context at the time of decision-making, but is also a function of the actual perception of road rules. Thus, our studies show risk perception, normative beliefs and declared behaviors differ according to the intentionality of pedestrian risky behaviors. In addition, pedestrians seem to differentiate between legal rules and social rules less than drivers do. This informal learning of the pedestrian behavior and the functioning of the road space is based on the behavioral consistency that the children observes, first in the family environment, then during their first accompanied and then autonomous pedestrian travels. The study of social norms is very important for several reasons. First, these are the first norms regarding road travel that the individual observes. Their learning will guide the perception of risk and the categorization of legal rules and therefore the behavior of the individual, throughout life. Second, these social norms govern the actual functioning of road space, more than the legal rules. Knowing these social norms allows users to understand and correctly anticipate the behavior of other users and to adapt to them safely. How to ensure that the autonomous vehicle is correctly understood and anticipated and that it correctly anticipates the behavior of users, who do not comply with legal standards? Taking into account social norms in the autonomous vehicle behavior model - at least temporarily, while waiting for a complete homogenization of the vehicle fleet - requires a clear understanding of their content. Observations will be necessary concerning driver behavior, but also users' behavior the autonomous vehicle has to anticipate. Quantitative and qualitative surveys of the norm content are also needed, particularly on the contextualized rules of use of the different travel modes.