Learning on graphs requires a graph feature representation able to discriminate among different graphs while being amenable to fast computation. The graph isomorphism problem tells us that no fast representation of graphs is known if we require the representation to be both invariant to nodes permutation and able to discriminate two non-isomorphic graphs. Most graph representations explored so far require to be invariant. We explore new graph representations by relaxing this constraint. We present a generic embedding of graphs relying on spectral graph theory called Invariant Graph Embedding (IGE). We show that for a large family of graphs, our embedding is still invariant. To evaluate the quality and utility of our IGE, we apply them to the graph classification problem and show that IGE reaches the state-of-the-art on benchmark datasets.