Advances in machine learning (ML) open the way to innovating functions in the avionic domain, such as navigation/surveillance assistance (e.g., vision-based navigation, obstacle sensing, virtual sensing), speech-to-text applications, autonomous flight, predictive maintenance, and cockpit assistance. Current standards and practices, which were defined and refined over decades with classical programming in mind, do not, however, support this new development paradigm. This paper provides an overview of the main challenges raised by the use of ML in the demonstration of compliance with regulatory requirements (i.e., software qualification) and an overview of literature relevant to these challenges, with particular focus on the issues of robustness, provability, and explainability of ML results.