Evaluating Human-Robot Interaction (HRI) is crucial for understanding the value that robots would bring to daily life. This paper investigates the robustness of machine learning classification techniques in interpreting physiological signals during HRI, considering potential artifacts induced by robot behavior. This phenomenon was explored with a 30 participants user study involving three cognitive efforts levels. This study used various physiological sensors, including Electroencephalography (EEG), Photoplethysmography (PPG), and Electrodermal Activity (EDA). Results reveal that EEG and PPG signal were impacted by robot-induced noise while EDA was not. By changing preprocessing parameters, EEG was also cleaned from robot noise and revealed better performances than EDA. The study highlights the importance of careful signal selection, balancing robustness and informativeness, and underscores the significance of preprocessing to ensure accurate classification aligned with users' mental states.