We consider a protocol for the two-time measurement of entropic observables in quantum open systems driven out of thermal equilibrium by coupling to several heat baths. We concentrate on the Markovian approximation of the time-evolution and relate the expected value of the so defined entropy variations with the well-known expression of entropy production due to Lebowitz and Spohn. We do so under the detailed balance condition and, as a byproduct, we show that the probabilities of outcomes of two-time measurements are given by a continuous time Markov process determined by the Lindblad generator of the Markovian quantum dynamics.