We provide an overview of continuous-time processes subject to jumps that do not originate from a compound Poisson process, but from a compound Hawkes process. Such stochastic processes allow for a clustering of self-exciting jumps, a phenomenon for which empirical evidence is strong. Our presentation, which omits certain technical details, focuses on the main ideas to facilitate applications to stochastic control theory. Among other things, we identify the appropriate infinitesimal generators for a set of problems involving various (possibly degenerate) cases of diffusions with self-exciting jumps. Compared to higher-dimensional diffusions, we note a degeneracy of the second-order infinitesimal generator. We derive a Feynman-Kac Theorem for a dynamic system driven by such a jump diffusion and also discuss a problem of continuous control of such a system and provide a verification theorem establishing a link between the value function and a novel type of Hamilton-Jacobi-Bellman (HJB) equation.