In the last decades, composite materials have been increasingly used in aircraft structures as a mean to reduce the weight. For safety reasons, periodic checks of impact-induced damage need to be performed. In that framework, ultrasound non destructive testing has proven its ability to detect macro-defects such as delamination. However, at early stages, Barely Visible Impact Damage (BVID) may not be detected via linear ultrasound techniques. In recent years, non-linear ultrasound has gained traction for the inspection of such defects. However, detection of the weak non-linear defect signature buried in ambient noise remains challenging, and depends on an empirical choice of excitation frequencies. This study focuses on the evaluation of a non-linear pump probe ultrasound inspection technique applied to BVID in multilayer carbon fibre reinforced composites used in aeronautics. In particular, the influence of various experimental parameters on the measured non-linear response is studied. A low frequency pump wave (via a shaker) and a high frequency wave (via a piezoelectric transducer) are transmitted to the medium while a second transducer records the ultrasonic response. Both pump and probe waves are sent in the form of a few secondslong frequency sweep, in the audible and ultrasonic range respectively. Appropriate post processing is then applied to the recorded signals to retrieve the non-linear response of the defect. The procedure is repeated on several composite plates with and without impact damages. The nonlinear response appears in the form of modulation sidebands in the frequency spectrum of the post-processed signals. The relative amplitudes of the side bands obtained for the various samples with various defect sizes and excitation amplitude or frequency content are studied. Optimal experimental parameters were obtained and led to a good detection of defects. The limits of the method are also discussed.