The long-range part of the interatomic interactions plays a substantial role in the collisional dynamics of ultracold gases. Here, we report on the calculation of the isotropic and anisotropic C6 coefficients characterizing the van der Waals interaction between dysprosium or erbium atoms in the two lowest energy levels and the ground-state alkali-metal (Li, Na, K, Rb, Cs, Fr) or alkaline-earth-metal (Be, Mg, Ca, Sr, Ba) atoms. The calculations are done using the integral of dynamic dipole polarizabilities at so-called imaginary frequencies of the two interacting atoms. For all atom pairs, we find that the isotropic C6 coefficients are two or three orders of magnitude larger than the anisotropic ones. Those coefficients are essential for modeling collisional properties of heteronuclear quantum mixtures containing highly magnetic dysprosium or erbium atoms and alkali-metal or alkaline-earth-metal atoms.