Separation of variables (SoV) is a special property of integrable models which ensures that the wavefunction has a very simple factorised form in a specially designed basis. Even though the factorisation of the wavefunction was recently established for higher rank models by two of the authors and G. Sizov, the measure for the scalar product was not known beyond the case of rank one symmetry. In this paper we show how this measure can be found, bypassing an explicit SoV construction. A key new observation is that the measure for spin chains in a highest-weight infinite dimensional representation of $\frak{sl}$(N) couples Q-functions at different nesting levels in a non-symmetric fashion. We also managed to express a large number of form factors as ratios of determinants in our new approach. We expect our method to be applicable in a much wider setup including the problem of computing correlators in integrable CFTs such as the fishnet theory, $ \mathcal{N} $ = 4 SYM and the ABJM model.