As touch screens become widely spread, making them more accessible to visually impaired people is an important task. Touch displays possess a poor accessibility for visually impaired people. One possibility to make them more accessible without sight is through gestural interaction. Yet, there are still few studies on using gestural interaction for visually impaired people. In this paper we present a comprehensive summary of existing projects investigating accessible gestural interaction. We also highlight the limits of current approaches and propose future working directions. Then, we present the design of an interactive map prototype that includes both a raised-line map overlay and gestural interaction for accessing different types of information (e.g., opening hours, distances). Preliminary results of our project show that basic gestural interaction techniques can be successfully used in interactive maps for visually impaired people.