Automated machine learning (AutoML) has transformed the process of selecting optimal machine learning (ML) models by autonomously searching for the most appropriate ones and fine-tuning associated hyperparameters. This eliminates the burdensome task of trial-and-error selection and parametrization of ML algorithms. Nonetheless, the lack of transparency and explainability poses a significant challenge when using AutoML, as it hampers user trust in the system’s recommendations. Consequently, users often allocate more resources to the search process, resulting in reduced efficiency of the AutoML systems. To address this challenge, we propose an interactive and explainable AutoML framework that enables users to understand the reasoning behind the recommendations and diagnose any limitations of the suggested models using various explainable AI methods. Additionally, our framework provides the possibility of automated performance refinement. To operationalize the framework, we introduce AMLExplainer, an XAI system for interactive and interpretable AutoML that visualizes and performs all stages of the proposed pipeline(s) within the widely used Bootstrap Dash environment.