Artificial Intelligence (AI) is envisioned to play a critical role in controlling and orchestrating 5G/IoT networks and their applications, thanks to its capabilities to recognize abnormal patterns in complex situations and produce accurate decisions. However, AI models are vulnerable to adversarial attacks, thus the societal view is far from trustworthy as to its usage in safety critical areas relying on 5G/IoT networks. In this paper, we present ongoing work being done in the H2020 SPATIAL project that targets developing and evaluating AI-based modules for anomaly detection and Root Cause Analysis in the 5G/IoT context regarding different criteria, such as explainability, resilience and performance on a real 5G/IoT testbed.