A basic issue in neurosciences is to look for possible relationships between brain architecture and cognitive models. The lack of architectural information in magnetic resonance images, however, has led the neuroimaging community to develop brain mapping strategies based on various coordinate systems without accurate architectural content. Therefore, the relationships between architectural and functional brain organizations are difficult to study when analyzing neuroimaging experiments. This paper advocates that the design of new brain image analysis methods inspired by the structural strategies often used in computer vision may provide better ways to address these relationships. The key point underlying this new framework is the conversion of the raw images into structural representations before analysis. These representations are made up of data-driven elementary features like activated clusters, cortical folds or fiber bundles. Two classes of methods are introduced. Inference of structural models via matching across a set of individuals is described first. This inference problem is illustrated by the group analysis of functional statistical parametric maps (SPMs). Then, the matching of new individual data with a priori known structural models is described, using the recognition of the cortical sulci as a prototypical example.