This article presents a time-domain numerical scheme for simulating EM-computational problems that include complex media and fine geometrical details in critical regions. These problems are common in engineering and applied physics. To name a few, microwave and optical devices that contain complex media, antennas characterization in presence of complex media, such as biological tissues, biomedical technology...etc. Modeling such scenarios requires the ability to handle two types of complexities, namely material and geometrical ones. Material complexity is taken into consideration using a convolution process with a matrix of time-domain filters; this models the dispersive and anisotropic nature of such media. On the other hand, non-conformal local mesh refinement approach is adopted to accurately discretize important fine details without exploding the computational resources. Numerical simulations are presented to show the efficiency, the accuracy and the stability of the proposed approach, with comparisons to FEM method and TLM method with regular fine meshing.