As a subclass of stochastic differential games with algebraic constraints, this article studies dynamic noncooperative games where the constraints are described by jump Markov differential-algebraic equations (DAEs). Theoretical tools, which require computing the infinitesimal generator and deriving Hamiton-Jacobi-Bellman equation for Markov jump DAEs, are developed. These fundamental results lead to pure feedback optimal strategies to compute the Nash equilibrium in noncooperative setting. In case of quadratic cost and linear dynamics, these strategies are obtained by solving coupled Riccati differential equations. The problem of robust control can be formulated as a two-player zero sum game and is solved by applying the results developed in this paper.