There are real strategic situations where nobody knows ex ante how many players there will be in the game at each step. Assuming that entry and exit could be modelized by random processes whose probability laws are common knowledge, we use dynamic programming and piecewise deterministic Markov decision processes to investigate such games. We study the dynamic equilibrium in games with randomly arriving players in discrete and continuous time for both finite and infinite horizon. Existence of dynamic equilibrium in discrete time is proved and we develop explicit algorithms for both discrete and continuous time linear quadratic problems. In both cases we offer a resolution for a Cournot oligopoly with sticky prices.