According to some authors, a task-irrelevant but highly salient stimulus will always capture attention at first, regardless of the Attentional Control Setting (ACS) defined by observer's goals. In this way, motion stimuli are known to be particularly salient and therefore may be easily selected when necessary but act as powerful distractors when irrelevant. Nevertheless, previous studies investigating the capacity of irrelevant motion stimuli to override an ACS for color produced conflicting results. The aim of our study was to compare to what extent focusing on motion can prevent a distraction effect by color and vice versa, when both dimensions compete for attentional selection. In Task 1, participants performed a visual search task for a target defined as a color-singleton while having to ignore an irrelevant motion-singleton. In Task 2, the instructions were reversed. Our results revealed a distraction effect in both tasks, suggesting that an ACS for a particular dimension is not sufficient to prevent attentional distraction by an irrelevant one. Moreover, our results showed a larger distraction effect for motion-distractors than for color-distractors. Our results are discussed in regard to current models of attentional control.