Automatic Feedback Generation for Short Answer Questions using Answer Diagnostic Graphs
Journal:
arXiv
Published Date:
Jan 27, 2025
Abstract
Short-reading comprehension questions help students understand text structure
but lack effective feedback. Students struggle to identify and correct errors,
while manual feedback creation is labor-intensive. This highlights the need for
automated feedback linking responses to a scoring rubric for deeper
comprehension.
Despite advances in Natural Language Processing (NLP), research has focused
on automatic grading, with limited work on feedback generation. To address
this, we propose a system that generates feedback for student responses.
Our contributions are twofold. First, we introduce the first system for
feedback on short-answer reading comprehension. These answers are derived from
the text, requiring structural understanding. We propose an "answer diagnosis
graph," integrating the text's logical structure with feedback templates. Using
this graph and NLP techniques, we estimate students' comprehension and generate
targeted feedback.
Second, we evaluate our feedback through an experiment with Japanese high
school students (n=39). They answered two 70-80 word questions and were divided
into two groups with minimal academic differences. One received a model answer,
the other system-generated feedback. Both re-answered the questions, and we
compared score changes. A questionnaire assessed perceptions and motivation.
Results showed no significant score improvement between groups, but
system-generated feedback helped students identify errors and key points in the
text. It also significantly increased motivation. However, further refinement
is needed to enhance text structure understanding.
