The numerical simulation of non-smooth hybrid systems exhibiting chattering behavior requires high computational costs. In the worst case, the simulation appears to come to a halt, since infinitely many discrete transitions would need to be simulated. In this paper we present an FMI-based framework and prototypical implementation for robust and reliable detection and elimination “On the Fly” of chattering behavior in run-time simulation of non-smooth hybrid systems. The main benefit of the developed framework is that it establishes solvability requirements and theorems for simulating hybrid systems while performing the chattering path avoidance internally in the master algorithm of the interface. This increases the efficiency of the chattering-free simulation as no enumeration of modes is required during the chattering detection and elimination process. The developed FMI-based framework can generate a chattering-free simulation for any generic chattering Functional Mockup Unit (FMU) conforming to the FMI standard v2.0 Specification for model exchange.