In this paper, we tackle the task of symbolic gesture recognition using inertial MicroElectroMechanicals Systems (MEMS) present in Smartphones. We propose to build a non-linear similarity metric based on a Siamese Neural Network (SNN), trained using a new error function that models the relations between pairs of similar and dissimilar samples in order to structure the network output space. Experiments performed on different datasets regrouping up to 22 individuals and 18 gesture classes, targeting the most likely real case applications, show that this structure allows for an improved classification and a higher rejection quality over the conventional MultiLayer Perceptron (MLP) and Dynamic Time Warping (DTW) similarity metric.