We review the fate of the Ostrogradsky ghost in higher-order theories. We start by recalling the original Ostrogradsky theorem and illustrate, in the context of classical mechanics, how higher-derivatives Lagrangians lead to unbounded Hamiltonians and then lead to (classical and quantum) instabilities. Then, we extend the Ostrogradsky theorem to higher-derivatives theories of several dynamical variables and show the possibility to evade the Ostrogradsky instability when the Lagrangian is ‘degenerate’, still in the context of classical mechanics. In particular, we explain why higher-derivatives Lagrangians and/or higher-derivatives Euler–Lagrange equations do not necessarily lead to the propagation of an Ostrogradsky ghost. We also study some quantum aspects and illustrate how the Ostrogradsky instability shows up at the quantum level. Finally, we generalize our analysis to the case of higher order covariant theories where, as the Hamiltonian is vanishing and thus bounded, the question of Ostrogradsky instabilities is subtler.