Comparing Student Models in Different Formalisms by Predicting their Impact on Help Success - Archive ouverte HAL Accéder directement au contenu
Communication Dans Un Congrès Année : 2013

Comparing Student Models in Different Formalisms by Predicting their Impact on Help Success

Résumé

We describe a method to evaluate how student models affect ITS decision quality - their raison d'être. Given logs of randomized tutorial decisions and ensuing student performance, we train a classifier to predict tutor decision outcomes (success or failure) based on situation features, such as student and task. We define a decision policy that selects whichever tutor action the trained classifier predicts in the current situation is likeliest to lead to a successful outcome. The ideal but costly way to evaluate such a policy is to implement it in the tutor and collect new data, which may require months of tutor use by hundreds of students. Instead, we use historical data to simulate a policy by extrapolating its effects from the subset of randomized decisions that happened to follow the policy. We then compare policies based on alternative student models by their simulated impact on the success rate of tutorial decisions. We test the method on data logged by Project LISTEN's Reading Tutor, which chooses randomly which type of help to give on a word. We report the cross-validated accuracy of predictions based on four types of student models, and compare the resulting policies' expected success and coverage. The method provides a utility-relevant metric to compare student models expressed in different formalisms.

Dates et versions

hal-00871563 , version 1 (15-10-2013)

Identifiants

Citer

Sébastien Lalle, Jack Mostow, Vanda Luengo, Nathalie Guin. Comparing Student Models in Different Formalisms by Predicting their Impact on Help Success. AIED 2013 - 16th International Conference on Artificial Intelligence in Education, Jul 2013, Memphis, TN, United States. pp.161-170, ⟨10.1007/978-3-642-39112-5_17⟩. ⟨hal-00871563⟩
389 Consultations
0 Téléchargements

Altmetric

Partager

Gmail Facebook X LinkedIn More