We consider the problem of distributed stochastic optimization in networks. Each node adjusts its action in order to optimize the global utility of the network, which is defined as the sum of local utilities of all nodes. Since the computation of the gradient may require much information exchange, we consider here that each node only has a noisy numerical observation of its local utility. This assumption is quite realistic, especially when the system is too complicated or constantly changing. At each time, nodes may exchange the observation of their numerical local utilities to estimate the global utility. Under the assumptions whether each node has collected the local utilities of all the other nodes or only part of these utilities, we propose two stochastic perturbation based distributed algorithms. We use tools from stochastic approximation to prove that both algorithms converge to the optimum. We then apply our algorithm to a power control problem in wireless networks and present numerical results that corroborate our claim.