Diffusion Magnetic Resonance Imaging (dMRI) is a powerful non-invasive and in-vivo imaging modality for probing brain white matter structure. Convolutional neural networks (CNNs) have been shown to be a powerful tool for many computer vision problems where the signals are acquired on a regular grid and where translational invariance is important. However, as we are considering dMRI signals that are acquired on a sphere, rotational invariance, rather than translational, is desired. In this work, we propose a spherical CNN model with fully spectral domain convolutional and non-linear layers. It provides rotational invariance and is adapted to the real nature of dMRI signals and uniform random distribution of sampling points. The proposed model is positively evaluated on the problem of estimation of neurite orientation dispersion and density imaging (NODDI) parameters on the data from Human Connectome Project (HCP).