Coin grading means evaluating coins’ physical condition. In this paper, we proposed a process for coin grading by quantification of "unexpected elements" such as scratches and dirty marks. We detect respectively significant and tiny “unexpected elements” with the help of handcrafted filters and Deep Learning techniques. The result of our process, which is close to the manual expert one, is considered as a useful help for numismatists.