The graph isomorphism (GI) problem asks whether two given graphs are isomorphic or not. The GI problem is quite basic and simple, however, it\textquoterights time complexity is a long standing open problem. The GI problem is clearly in NP, no polynomial time algorithm is known, and the GI problem is not NP-complete unless the polynomial hierarchy collapses. In this paper, we survey the computational complexity of the problem on some graph classes that have geometric characterizations. Sometimes the GI problem becomes polynomial time solvable when we add some restrictions on some graph classes. The properties of these graph classes on the boundary indicate us the essence of difficulty of the GI problem. We also show that the GI problem is as hard as the problem on general graphs even for grid unit intersection graphs on a torus, that partially solves an open problem.