We have measured the short-time and long-time elastic responses to compression of various types of surfactant layers adsorbed at oil-water interfaces. We prepared reasonably monodisperse oil-in-water emulsions with the same surfactants and monitored the time evolution of the emulsion droplets' diameter.Weused a broad variety of surfactants (cationic, nonionic, and small polymers) and alkanes with different chain lengths. The emulsion drop size evolution is first controlled by Ostwald ripening and later on by drop coalescence, the later step being quite short. The overall emulsion lifetime is therefore dominated by ripening and for a given oil appears well correlated with the low-frequency surface elasticity as expected (and not with the high-frequency one, which is expected to control coalescence). When the oil chain length is changed, the stability is related more to the oil solubility in water, which also controls ripening. The overall results demonstrate the great importance of surface elasticity in emulsion stability.