Communication Dans Un Congrès Année : 2026

Exploring the Impact of LLM-Based Scaffolding on Academic Performance and the Mediating Roles of AI Literacy and Prior Knowledge

Résumé

Large Language Models (LLMs) have rapidly gained popularity among university students for tasks such as essay writing, virtual tutoring, and coding exercises. Although the number of studies applying LLM-based solutions in educational contexts has grown significantly, researchers emphasize the need for empirical studies to evaluate the effectiveness of these tools to support teaching and learning. To address this gap, this paper presents a study analyzing the effects of an LLM-based system designed to support students’ self-reflection. The study compares the grades and course outcomes of two groups of students enrolled in the same Thermodynamics course during two academic years (2023 and 2024, 234 students in total), but only the 2024 cohort had access to an LLM-based chatbot specifically designed to support self-reflection. The data showed no significant correlation between students’ AI literacy profiles, their prior experience with generative AI, and the adoption of the tool. However, an analysis of engagement with the tool in the 2024 cohort revealed that students who interacted more extensively with the chatbot—particularly medium and high achievers (based on prior academic performance)—demonstrated a significant improvement in their final grades.

Fichier non déposé

Dates et versions

hal-05294878 , version 1 (02-10-2025)

Identifiants

Citer

André Nascimento, Mar Pérez-Sanagustín, Jean-François Parmentier, Isabel Hilliger, Gabriel Ferrettini, et al.. Exploring the Impact of LLM-Based Scaffolding on Academic Performance and the Mediating Roles of AI Literacy and Prior Knowledge. Twentieth European Conference on Technology Enhanced Learning ECTEL 2025, EATEL, Sep 2025, Newcastle and Durham, UK, France. pp.382-396, ⟨10.1007/978-3-032-03870-8_26⟩. ⟨hal-05294878⟩
122 Consultations
0 Téléchargements

Altmetric

Partager

  • More