Recently, the civilian applications of Unmanned Aerial Vehicles (UAVs) are gaining more interest in several domains. Due to operational costs, safety concerns, and legal regulations, Agent-Based Simulation (ABS) is commonly used to design models and conduct tests. This has resulted in numerous research works addressing ABS in civilian UAV applications. This paper aims to provide a comprehensive overview of the ABS contribution in civilian UAV applications by conducting a Systematic Literature Review (SLR) on the relevant research in the previous ten years. Following the SLR methodology, this objective is broken down into several research questions aiming to (i) understand the evolution of ABS use in civilian UAV applications and identify the related hot research topics, (ii) identify the underlying artificial intelligence systems used in the literature, (iii) understand how and when ABS is integrated in broader and more complex internet of things & ubiquitous computing environments, and (iv) identity the communication technologies, tools, and evaluation techniques used to design, implement, and test the proposed ABS models. From the SLR results, key research directions are highlighted including problems related to autonomy, explainability, security, flight duration, integration within smart cities, regulations, and validation & verification of the UAV behavior.