For reversible Markov chains on finite state spaces, we show that the modified log-Sobolev inequality (MLSI) can be upgraded to a log-Sobolev inequality (LSI) at the surprisingly low cost of degrading the associated constant by log(1/p), where p is the minimum non-zero transition probability. We illustrate this by providing the first log-Sobolev estimate for Zero-Range processes on arbitrary graphs. As another application, we determine the modified log-Sobolev constant of the Lamplighter chain on all bounded-degree graphs, and use it to provide negative answers to two open questions by Montenegro and Tetali (2006) and Hermon and Peres (2018). Our proof builds upon the regularization trick' recently introduced by the last two authors.