Understanding the mapping between structural and functional brain connectivity is essential for understanding how cognitive processes emerge from their morphological substrates. Many studies have investigated the problem from an eigendecomposition viewpoint, however, few have taken a deep learning viewpoint, even less studies have been engaged within the framework of graph neural networks (GNNs). As deep learning has produced significant results in several fields, there has been an increasing interest in applying neural networks to graph problems. In this paper, we investigate the structural connectivity and functional connectivity mapping within a deep learning GNNs based framework, including graph convolutional networks (GCN) and graph transformer networks (GTN). To our knowledge, this original GTN based framework has never been studied in the context of structure-function and brain connectivity mapping. To achieve this goal, we use a GNNs based encoder-decoder system, where the encoder takes structural connectivity (SC) matrix as input and generates a latent representation of each node in a lower dimension, then the decoder uses the latent representation to reconstruct or predict the associated functional connectivity (FC) matrix. Besides comparing different encoders for node embedding, we also demonstrate that a decoder, which projects lower dimension vectors onto higher dimensional space, can improve the model performance. Our experiments demonstrate that both GCN encoder and GTN encoder combined with the proposed decoder can provide better results on our data than the previously proposed GCN autoencoder model. GTN encoder is also shown to be much more effective when it comes to noisy data and outliers.