Arc-Kayles is a game where two players alternate removing two adjacent vertices until no move is left. Introduced in 1978, its computational complexity is still open. More recently, subtraction games, where the players cannot disconnect the graph while removing vertices, were introduced. In particular, Arc-Kayles admits a non-disconnecting variant that is a subtraction game. We study the computational complexity of subtraction games on graphs, proving that they are PSPACE-complete even on very structured graph classes (split, bipartite of any even girth). We prove that Non-Disconnecting Arc-Kayles can be solved in polynomial-time on unicyclic graphs, clique trees, and subclasses of threshold graphs. We also show that a sufficient condition for a second player-win on Arc-Kayles is equivalent to the graph isomorphism problem.