This study is part of an ongoing larger project concerning student self-assessment skills in university courses. We have developed a method enabling large cohorts of students to assess their own learning outcomes and to give their own course grades with the help of an automatic verification system. This paper explores the question of accuracy, namely, whether the self-assessed grades correspond to the students’ actual skills, and how well the automatic system can pick up issues in the self-assessment. Based on an expert’s evaluation of the skills of two students, we conclude that although for large part the model works as intended, there are some cases where neither the self-assessment nor the computer verification seem to be accurate.