Detection of B-mode polarization of the cosmic microwave background (CMB) radiation is one of the frontiers of observational cosmology. Because they are an order of magnitude fainter than E-modes, it is quite a challenge to detect B-modes. Having more manageable systematics, interferometers prove to have a substantial advantage over imagers in detecting such faint signals. Here, we present a method for Bayesian inference of power spectra and signal reconstruction from interferometric data of the CMB polarization signal by using the technique of Gibbs sampling. We demonstrate the validity of the method in the flat-sky approximation for a simulation of an interferometric observation on a finite patch with incomplete uv-plane coverage, a finite beam size, and a realistic noise model. With a computational complexity of O(n ^3/2), n being the data size, Gibbs sampling provides an efficient method for analyzing upcoming cosmology observations.