We introduce the "Engineering Dwarfs at Galaxy formation's Edge" (EDGE) project to study the cosmological formation and evolution of the smallest galaxies in the Universe. In this first paper, we explore the effects of resolution and sub-grid physics on a single low mass halo (M halo = 10 9 M), simulated to redshift z = 0 at a mass and spatial resolution of ∼ 20 M and ∼ 3 pc. We consider different star formation prescriptions, supernova feedback strengths and on-the-fly radiative transfer (RT). We show that RT changes the mode of galactic self-regulation at this halo mass, suppressing star formation by causing the interstellar and circumgalactic gas to remain predominantly warm (∼ 10 4 K) even before cosmic reionisation. By contrast, without RT, star formation regulation occurs only through starbursts and their associated vigorous galactic outflows. In spite of this difference, the entire simulation suite (with the exception of models without any feedback) matches observed dwarf galaxy sizes, velocity dispersions, V-band magnitudes and dynamical mass-to-light-ratios. This is because such structural scaling relations are predominantly set by the host dark matter halo, with the remaining model-to-model variation being smaller than the observational scatter. We find that only the stellar mass-metallicity relation differentiates the galaxy formation models. Explosive feedback ejects more metals from the dwarf, leading to a lower metallicity at a fixed stellar mass. We conclude that the stellar mass-metallicity relation of the very smallest galaxies provides a unique constraint on galaxy formation physics.