In single-cell transcriptomics, machine learning techniques have been applied for automatic cell annotation [1], for the identification of cancer cell subpopulations [2], and for modelling the transcriptional dynamics that govern cellular fate and development [3-4]. These methods hold potential value for routine practice in clinical settings but must address critical challenges in the use-and misuse-of AI algorithms for reliability and interpretability [5]. The field of explainable AI addresses these concerns, among which the robustness to adversarial attacks, i.e. techniques designed to fool a machine learning model with deceptive and inaccurate data. Here we present adverSCarial, an original R package that generates adversarial attacks on single-cell transcriptomics classifiers (https://github.com/GhislainFievet/adverSCarial). adverSCarial currently proposes four customizable functions to produce adversarial attacks. In this work, we define two types of methods: the minimal (min) and the maximal (max) change attacks. The min change attack finds the smallest possible perturbation in the input data leading to a change of classification. The max change attack finds the largest data modification which does not alter the initial classification. On a reference peripheral blood mononuclear cells (PBMC) dataset of 2,700 cells and 22,042 genes [6], we further tested adverSCarial and compared the susceptibility of two published cell type classifiers to these attacks, the classification tree based CHETAH [7] and the marker based scType [8]. Both classifiers showed weaknesses to min and max adversarial attacks, especially to the max change method. Indeed, we found that it is possible to modify a significant proportion of genes without affecting the classification confidence. For a representative example, on the CD14 monocytes cluster of the studied PBMC dataset, replacing all 22,042 gene expression values by their last percentile did not modify CHETAH identification as CD14 cells. These results were generalized to all cell types and highlight the concern that machine learning algorithms may fail to detect even significant anomalies in the input data. In conclusion, this work demonstrates the usefulness of such techniques in testing the robustness of classifier families and the extent of the modifications required to deviate their intended use. We believe that our approach and tool can guide the development and validation of more reliable models that could be used in clinical setups, and aim to extensively evaluate these tools on a wide variety of single-cell datasets.