Gaussian process (GP) models have become a well-established framework for the adap-tive design of costly experiments, and in particular, but not only, of computer experiments. GP-based sequential designs have been proposed for various objectives, such as global optimization (estimating the global maximum or maximizer(s) of a function), reliability analysis (estimating a probability of failure) or the estimation of level sets and excursion sets. In this paper, we tackle the convergence properties of an important class of such sequential design strategies, known as stepwise uncertainty reduction (SUR) strategies. Our approach relies on the key observation that the sequence of residual uncertainty measures, in a SUR strategy, is in general a supermartingale with respect to the filtration generated by the observations. We also provide some general results about GP-based sequential design, which are of independent interest. Our main application is a proof of almost sure convergence for one of the SUR strategies proposed by Bect, Ginsbourger, Li, Picheny and Vazquez (Stat. Comp., 2012). To the best of our knowledge, this is the first convergence proof for a GP-based sequential design algorithm dedicated to the estimation of probabilities of excursion and excursions sets. We also establish, using the same approach, a new proof of almost sure convergence for the expected improvement algorithm, which is the first proof for this algorithm that applies to any continuous GP.