The tutorial provides an introduction to the capabilities of software environment Marmote via its Python interface: pyMarmote/pyMarmoteMDP. Marmote is a programming library for modeling with Markov chains, analyzing and “solving” these chains. It provides the objects for building continuous-time and discrete-time Markov chains on discrete but possibly complicated state spaces. Once defined, Markov chains can be analyzed with a variety of methods, including structural analysis, Monte-Carlo simulation and numerical solution for criteria such as transient and stationary distributions, or average hitting times. The extention MarmoteMDP provides a library for modeling with Markov Decision Processes. It provides algorithms for numerically determining optimal policies for all classical optimization criteria. It also features capabilities for the structural analysis of the resulting policies and value functions. The presentation consists in showing how to use Marmote in a series of thematic Python notebooks. These notebooks will be shared so as to allow motivated attendants to practice themselves.