Deep Learning (DL) technologies have recently gained significant attention and have been applied to Network Intrusion Detection Systems (NIDS). However, DL is known to be vulnerable to adversarial attacks, which evade detection by introducing perturbations to input data. Meanwhile, eXplainable Artificial Intelligence (XAI) helps us to understand predictions made by DL models and is an essential technology for ensuring accountability. This paper focuses on the relationship between the DL model’s decision-making processes and adversarial examples (AEs) and proposes a new AE generation method based on XAI. Our method utilizes XAI to identify important features when making predictions and perturb them in real (traffic) space to evade detection by DL-based NIDS. We implemented our proposed method in a real-world network environment. We confirmed that our AEs completely evade detection without compromising the malicious nature of the attack communications. This experiment reveals that, unlike many existing studies, our proposed method is feasible in the traffic space.