We study the evolution of gene frequencies in a spatially distributed population when the dispersal of individuals is not uniform in space. We adapt the spatial Λ-Fleming-Viot process to this setting and consider that individuals spread their offspring farther from themselves at each generation in one halfspace than in the other. We study the large scale behaviour of this process and show that the motion of ancestral lineages is asymptotically close to a family of skew Brownian motions which coalesce upon meeting in one dimension, but never meet in higher dimension. This leads to a generalization of a result due to Nagylaki on the scaling limits of the gene frequencies: the non-uniform dispersal causes a discontinuity in the slope of the gene frequencies but the gene frequencies themselves are continuous across the interface.