Polycrystalline aggregates lacking four independent systems for the glide of dislocations can deform in a purely viscoplastic regime only if additional deformation mechanisms (such as grain boundary sliding, diffusion,. . .) are activated. We introduce an implementation of the self-consistent scheme in which this additional physical mechanism, considered as a stress relaxation mechanism, is represented by a nonlinear isotropic vis-coplastic potential. Several nonlinear extensions of the self-consistent scheme, including the second-order method of Ponte-Castañeda, are used to provide an estimate of the effective viscoplastic behavior of such polycrystals. The implementation of the method includes an approximation of the isotropic potential to ensure convergence of the attractive fixed-point numerical algorithm. The method is then applied to olivine polycrystals, the main constituent of the Earth's upper mantle. Due to the extreme local anisotropy of the local constitutive behavior and the subsequent intraphase stress and strain-rate field heterogeneities, the second-order method is the only extension providing qualitative and quantitative accurate results. The effective viscosity is strongly dependent on the strength of the relaxation mechanism. For olivine, a linear viscous relaxation (e.g. diffusion) could be relevant; in that case, the polycrystal stress sensitivity is reduced compared to that of dislocation glide, and the most active slip system is not necessarily the one with the smallest reference stress due to stress concentrations. This study reveals the significant importance of the strength and stress sensitivity of the additional relaxation mechanism for the rheology and lattice preferred orientation in such highly anisotropic polycrystalline aggregates.