Sensory cortices modulate innate behaviors through corticofugal projections targeting phylogenetically-old brainstem nuclei. However, the principles behind the functional connectivity of these projections remain poorly understood. Here, we show that in mice visual cortical neurons projecting to the optic-tract and dorsal-terminal nuclei (NOT-DTN) possess distinct response properties and anatomical connectivity, supporting the adaption of an essential innate eye movement, the optokinetic reflex (OKR). We find that these corticofugal neurons are enriched in specific visual areas, and they prefer temporo-nasal visual motion, matching the direction bias of downstream NOT-DTN neurons. Remarkably, continuous OKR stimulation selectively enhances the activity of these temporo-nasally biased cortical neurons, which can efficiently promote OKR plasticity. Lastly, we demonstrate that silencing downstream NOT-DTN neurons, which project specifically to the inferior olive—a key structure in oculomotor plasticity, impairs the cortical modulation of OKR and OKR plasticity. Our results unveil a direction-selective cortico-brainstem pathway that adaptively modulates innate behaviors.