Artificial Neural networks (ANNs) have become essential components in various safety-critical applications, including autonomous vehicles, medical devices, and avionics, where system failures can lead to severe risks. Edge AI devices, which process data locally without relying on the cloud, are increasingly used to meet the performance and real-time demands of these applications. However, their reliability in radiation-prone environments is a significant concern. In this context, this paper evaluates the MAX78000, an ultra-low-power Edge AI microcontroller with a hardware-based Convolutional Neural Network (CNN) accelerator, focusing on its behavior in radiation environments. To assess the reliability of the MAX78000, we performed a test campaign at the ChipIR neutron irradiation facility using two different ANNs. We implemented techniques to improve system observability during ANN inference and analyzed the radiation-induced errors observed. The results present a comparative analysis between the two ANN architectures, which shows that the complexity of the ANN directly impacts its reliability.