The portal connecting the invisible and visible sectors is one of the most natural explanations of the dark world. However, the early-time dark matter production via the portal faces extremely stringent late-time constraints. To solve such tension, we construct the scalar-controlled kinetic mixing varying with the ultralight CP-even scalar's cosmological evolution. To realize this and eliminate the constant mixing, we couple the ultralight scalar within the mass range $10^{-33}\text{eV} \lesssim m_0 \ll \text{eV}$ with the heavy doubly charged messengers and impose the $\mathbb{Z}_2$ symmetry under the dark charge conjugation. Via the varying mixing, the $\text{keV}-\text{MeV}$ dark photon dark matter is produced through the early-time freeze-in when the scalar is misaligned from the origin and free from the late-time exclusions when the scalar does the damped oscillation and dynamically sets the kinetic mixing. We also find that the scalar-photon coupling emerges from the underlying physics, which changes the cosmological history and provides the experimental targets based on the fine-structure constant variation and the equivalence principle violation. To protect the scalar naturalness, we discretely re-establish the broken shift symmetry by embedding the minimal model into the $\mathbb{Z}_N$-protected model. When $N \sim 10$, the scalar's mass quantum correction can be suppressed much below $10^{-33}\text{eV}$. Moreover, utilizing two different methods, we precisely calculate the $\mathbb{Z}_N$-invariant Coleman-Weinberg potential to arbitrary orders. Finally, we explore the varying kinetic mixing in the Dirac gaugino model and the dark matter models via other varying portals.