Gravity is a major cue for the proper growth and development of plants. The response of plants to gravity implies starch-filled plastids, the statoliths, which sediments at the bottom of the gravisensing cells, the statocytes. Statoliths are assumed to modify the transport of the growth hormone, auxin, by acting on specific auxin transporters, PIN proteins. However, the complete gravitropic signaling pathway from the intracellular signal associated to statoliths to the plant bending is still not well understood. In this article, we build on recent experimental results showing that statoliths do not act as gravitational force sensor, but as position sensor, to develop a bottom-up theory of plant gravitropism. The main hypothesis of the model is that the presence of statoliths modifies PIN trafficking close to the cell membrane. This basic assumption, coupled with auxin transport and growth in an idealized tissue made of a one-dimensional array of cells, recovers several major features of the gravitropic response of plants. First, the model provides a new interpretation for the response of a plant to a steady stimulus, the so-called sine-law of plant gravitropism. Second, it predicts the existence of a gravity-independent memory process as observed recently in experiments studying the response to transient stimulus. The model suggests that the timescale of this process is associated to PIN turnover, calling for new experimental studies.