The femoral stem primary stability achieved by the impaction of an ancillary during its insertion is an important factor of success in cementless surgery. However, surgeons still rely on their proprioception, making the process highly subjective. The use of Experimental Modal Analysis (EMA) without sensor nor probe fixation on the implant or on the bone is a promising non destructive approach to determine the femoral stem stability. The aim of this study is to investigate whether EMA performed directly on the ancillary could be used to monitor the femoral stem insertion into the bone. To do so, a cementless femoral stem was inserted into 10 bone phantoms of human femurs and EMA was carried out on the ancillary using a dedicated impact hammer for each insertion step. Two bending modes could be identified in the frequency range [400–8000] Hz for which the resonance frequency was shown to be sensitive to the insertion step and to the bone-implant interface properties. A significant correlation was obtained between the two modal frequencies and the implant insertion depth (R2 = 0.95 ± 0.04 and R2 = 0.94 ± 0.06). This study opens new paths towards the development of noninvasive vibration based evaluation methods to monitor cementless implant insertion.