Phonological Awareness (PA) is defined as “one’s ability to recognize, discriminate, and manipulate the sounds in one’s language, regardless of the size of the word unit that is the focus” (Anthony & Francis, 2005). PA, and specifically phonemic awareness, plays a crucial role in L1 and L2 reading acquisition (Snow et al., 2001 ; Haigh et al., 2011) and L2 pronunciation (Kivistö-de Souza, 2015), but its relation to L2 listening has been studied to a lesser degree (Li et al, 2012). Furthermore, various authors defend the hypothesis that PA is not linked to a specific language but rather a general linguistic meta-skill (Comeau et al., 1999; Cummins, 1979). In order to study the relationship between the development of PA and the use of an English app targeting listening skills, a reliable test was needed (Charles et al., 2022). Previous studies of PA have either used ad hoc tests which are difficult to use in other contexts, or commercial tests already validated and calibrated with a native population (e.g. CTOPP, Wagner, Torgesen & Rashotte, 1999). Using these tests with non native child learners has proven difficult, because of a lack of lexical knowledge, leading to items of questionable reliability (due to children guessing). Moreover, these tests are usually paper-based, and designed to be administered individually by trained professionals or research assistants. They are thus costly and time-consuming to use, and difficult to access for both teachers and researchers. The goal of this study is to design a PA test to be administered on tablets, taken by French children autonomously and assessed automatically. This requires the following design features: - the tasks need to be transformed into selected response tasks to be automatically scorable; - each task must start with several training items; - in order for instructions to be comprehensible to young learners, they must be carefully scripted, illustrated and contextualized (in our case, with a family of monsters enjoying words starting or ending with certain sounds); - instructions should not use written text. The application was tested with 65 intact classes of 1st graders (6-7 years old) learning English in France, whose schools participated in a larger project focused on L1 French reading fluency and L2 English listening comprehension. We will describe the six tasks used in L1 French and L2 English, and present the analysis of the results. The test will be analyzed for reliability, and the items for their discriminatory power. Results of classroom observations during administration, teacher feedback and user traces will also be considered. Finally, we will explain what remains to be done to make the application available to teachers and researchers.