We consider the problem of coordinated multi-cell downlink beamforming in massive multiple input multiple output (MIMO) systems consisting of N cells, Nt antennas per base station (BS) and K user terminals (UTs) per cell. Specifically, we formulate a multi-cell beamforming algorithm for massive MIMO systems which requires limited amount of information exchange between the BSs. The design objective is to minimize the total transmit power across all the BSs subject to satisfying the user signal to interference noise ratio (SINR) constraints. With our algorithm, the BSs need to exchange parameters which can be computed solely based on the channel statistics rather than the instantaneous CSI. We make use of tools from random matrix theory to formulate the decentralized algorithm. We also characterize a lower bound on the set of target SINR values for which the decentralized multi-cell beamforming algorithm is feasible. We further show that the performance of our algorithm asymptotically matches the performance of the centralized algorithm with full CSI sharing. Finally, we investigate the impact of imperfect CSI and pilot contamination effect on the performance of the decentralized algorithm, and propose a heuristic extension of the algorithm to accommodate these issues. Simulation results illustrate that our algorithm closely satisfies the target SINR constraints and achieves minimum power in the regime of massive MIMO systems. In addition, it also provides substantial power savings as compared to zero-forcing beamforming when the number of antennas per BS is of the same orders of magnitude as the number of UTs per cell.