Traits under migration-selection balance are increasingly shown to exhibit complex patterns of genetic architecture, with allelic differences of varying magnitude. However, studying the influence of a large number of small allelic effects on the maintenance of spatial polymorphism is mathematically challenging, due to the high complexity of the systems that arise. Here we propose a new methodology that allows us to take into account the combined contributions of a major locus and of a quantitative background resulting from small effect loci, inherited according to the infinitesimal model. In a regime of small variance contributed by the quantitative loci, we found new arguments of convex analysis to justify that traits are concentrated around the major alleles effects according to a normal distribution, which leads to a slow-fast analysis approach. By applying it to a symmetrical two patch model, we predict an undocumented phenomenon of loss of polymorphism at the major locus despite strong selection for local adaptation under some conditions, where the infinitesimal quantitative background slowly disrupts the fast established symmetrical polymorphism at the major locus, which is confirmed by individual-based simulations. We also provide a comprehensive toolbox designed to describe how to apply our method to more complex population genetic models.