In this paper, we present a framework to reconstruct spatially localized sources from Magnetoencephalogra-phy (MEG)/Electroencephalography (EEG) using spatiotempo-ral constraint. The source dynamics are represented by a Mul-tivariate Autoregressive (MAR) model whose matrix elements are constrained by the anatomical connectivity obtained from diffusion Magnetic Resonance Imaging (dMRI). The framework assumes that the whole brain dynamic follows a constant MAR model in a time window of interest. The source activations and the MAR model parameters are estimated iteratively. We could confirm the accuracy of the framework using simulation experiments in both high and low noise levels. The proposed framework outperforms the two-stage approach.