Improving Automated Evaluation of Student Text Responses Using GPT-3.5 for Text Data Augmentation - Archive ouverte HAL
Communication Dans Un Congrès Année : 2023

Improving Automated Evaluation of Student Text Responses Using GPT-3.5 for Text Data Augmentation

Keith Cochran
Clayton Cohn
Peter Hastings

Résumé

In education, intelligent learning environments allow students to choose how to tackle open-ended tasks while monitoring performance and behavior, allowing for the creation of adaptive support to help students overcome challenges. Timely feedback is critical to aid students' progression toward learning and improved problem-solving. Feedback on text-based student responses can be delayed when teachers are overloaded with work. Automated evaluation can provide quick student feedback while easing the manual evaluation burden for teachers in areas with a high teacher-to-student ratio. Current methods of evaluating student essay responses to questions have included transformer-based natural language processing models with varying degrees of success. One main challenge in training these models is the scarcity of data for studentgenerated data. Larger volumes of training data are needed to create models that perform at a sufficient level of accuracy. Some studies have vast data, but large quantities are difficult to obtain when educational studies involve student-generated text. To overcome this data scarcity issue, text augmentation techniques have been employed to balance and expand the data set so that models can be trained with higher accuracy, leading to more reliable evaluation and categorization of student answers to aid teachers in the student's learning progression. This paper examines the text-generating AI model, GPT-3.5, to determine if prompt-based text-generation methods are viable for generating additional text to supplement small sets of student responses for machine learning model training. We augmented student responses across two domains using GPT-3.5 completions and used that data to train a multilingual BERT model. Our results show that text generation can improve model performance on small data sets over simple self-augmentation.
Fichier principal
Vignette du fichier
Cochran et al 2023 AIEd preprint.pdf (242.02 Ko) Télécharger le fichier
Origine Fichiers produits par l'(les) auteur(s)

Dates et versions

hal-04304851 , version 1 (29-11-2023)

Identifiants

Citer

Keith Cochran, Clayton Cohn, Jean-François Rouet, Peter Hastings. Improving Automated Evaluation of Student Text Responses Using GPT-3.5 for Text Data Augmentation. 24th International Conference Artificial Intelligence in Education (AIED 2023), Jul 2023, Tokyo, Japan. pp.217-228, ⟨10.1007/978-3-031-36272-9_18⟩. ⟨hal-04304851⟩
54 Consultations
394 Téléchargements

Altmetric

Partager

More