The emerging integration of IoT (Internet of Things) and AI (Artificial Intelligence) has unlocked numerous opportunities for innovation across diverse industries. However, growing privacy concerns and data isolation issues have inhibited this promising advancement. Unfortunately, traditional centralized machine learning (ML) methods have demonstrated their limitations in addressing these hurdles. In response to this ever-evolving landscape, Federated Learning (FL) has surfaced as a cutting-edge machine learning paradigm, enabling collaborative training across decentralized devices. FL allows users to jointly construct AI models without sharing their local raw data, ensuring data privacy, network scalability, and minimal data transfer. One essential aspect of FL revolves around proficient knowledge aggregation within a heterogeneous environment. Yet, the inherent characteristics of FL have amplified the complexity of its practical implementation compared to centralized ML. This survey delves into three prominent clusters of FL research contributions: personalization, optimization, and robustness. The objective is to provide a well-structured and fine-grained classification scheme related to these research areas through a unique methodology for selecting related work. Unlike other survey papers, we employed a hybrid approach that amalgamates bibliometric analysis and systematic scrutinizing to find the most influential work in the literature. Therefore, we examine challenges and contemporary techniques related to heterogeneity, efficiency, security, and privacy. Another valuable asset of this study is its comprehensive coverage of FL aggregation strategies, encompassing architectural features, synchronization methods, and several federation motivations. To further enrich our investigation, we provide practical insights into evaluating novel FL proposals and conduct experiments to assess and compare aggregation methods under IID and non-IID data distributions. Finally, we present a compelling set of research avenues that call for further exploration to open up a treasure of advancement.