Second language (L2) learners frequently encounter difficulty in perceiving specific non-native sound contrasts (Dupoux et al., 2008), i.e. a phenomenon called phonological deafness (Troubetzkoi, 1939). However, if the neurocognitive network underlying phonological processing is plastic to some extent, extensive L2 experience should lead to adaptive processes and eventually to a certain capacity to discriminate non-native phonemic contrasts even in late L2 learners (Best & Strange, 1992; Flege et al., 1997; Iverson et al., 2012). Here, our goal was to examine the extent to which neuroplastic changes take place in the phonological system as a function of L2 experience. We designed an ERP experiment in which the capacity of listeners to discriminate second language phonemic contrasts mediated lexical access. A semantic violation paradigm was used in which the difference between semantically congruent and incongruent items was implemented by a phonemic contrast that was unique to the L2, English (e.g., /ɪ/ - /i/: ship – sheep). Nineteen young adult native speakers of French with intermediate proficiency in English participated in the ERP experiment. Participants listened to sentences that contained either a semantically congruent item (e.g., The anchor of the ship was let down) or an incongruent one (e.g., *The anchor of the sheep was let down) and were asked to perform an acceptability judgement. The ERP data revealed a fronto-central incongruency effect, i.e. a larger negativity for incongruent than congruent words between 180-220 ms after critical word onset. Importantly, this early effect was larger in more proficient L2 learners. Moreover, a centro-parietal N400 incongruency effect was found, i.e. larger negativity for incongruent than congruent words between 350-650 ms. Within this time window, the effect peaked earlier in more proficient L2 learners. The present findings indicate that L2 learners were sensitive to semantic incongruencies mediated by non-native phonemic contrasts. The proficiency-related variations of ERP effects suggest that perceptual sensitivity to non-native sounds depends on the amount and type of sustained non-native language experience. However, the capacity to acquire non-native phonology has previously been found to show considerable inter-individual variability (Pruitt et al., 2006; Golestani & Zatorre, 2009). Thus, in the future it will be of importance to delimitate facilitating and limiting factors for neuroplastic changes in the phonological system. Amongst others, investigations should identify how targeted training can improve the sensitivity to second language phonemic contrasts.