This paper investigates a new approach to approximate semi-discrete optimal transport for large-scale problem, i.e. in high dimension and for a large number of points. The proposed technique relies on a hierarchical decomposition of the target discrete distribution and the transport map itself. A stochastic optimization algorithm is derived to estimate the parameters of the corresponding multi-layer weighted nearest neighbor model. This model allows for fast evaluation during synthesis and training, for which it exhibits faster empirical convergence. Several applications to patch-based image processing are investigated: texture synthesis, texture inpainting, and style transfer. The proposed models compare favorably to the state of the art, either in terms of image quality, computation time, or regarding the number of parameters. Additionally, they do not require any pixel-based optimization or training on a large dataset of natural images.