Multi-axis milling and other computer numerical control machining processes allow us to create very complex geometries and thin parts. In this context, virtual machining is a powerful tool, but the simultaneous vibrations of the tool and workpiece are not easy to define and take into account. This paper presents a general method with which to simulate material removal when both the workpiece and tool are assumed to be non-rigid. We consider that they both vibrate when we define the Boolean chip. This is not usually considered with the aim of predicting the machined surface vibrations and the resulting geometric defects including roughness. By extending the material frame associated with the non-rigid workpiece, our method precisely defines the material removal for any tool or workpiece. It then allows us to establish a method of deriving efficient numerical approximations with which to simulate a succession of machining operations from roughening to finishing. Two kinds of finite element approximations are linked. One is a classical elastic finite element model including damping. The second, which is kinematically linked to the first, accurately describes the relative motion of each part of the tool with respect to the workpiece, and ensures the description of material removal and related forces. Two industrial examples show the potential of the method.