An Evaluation of a Hybrid Language Understanding Approach for Robust Selection of Tutoring Goals
In IJAIED
15 (1)
Publication information
Abstract
In this paper, we explore the problem of selecting appropriate interventions for students based
on an analysis of their interactions with a tutoring system. In the context of the WHY2 conceptual physics
tutoring system, we describe CarmelTC, a hybrid symbolic/statistical approach for analysing conceptual
physics explanations in order to determine which Knowledge Construction Dialogues (KCDs) students
need for the purpose of encouraging them to include important points that are missing. We briefly describe
our tutoring approach. We then present a model that demonstrates a general problem with selecting
interventions based on an analysis of student performance in circumstances where there is uncertainty with
the interpretation, such as with speech or text based natural language input, complex and error prone
mathematical or other formal language input, graphical input (i.e., diagrams, etc.), or gestures. In
particular, when student performance completeness is high, intervention selection accuracy is more
sensitive to analysis accuracy, and increasingly so as performance completeness increases. In light of this model, we have evaluated our CarmelTC approach and have demonstrated that it performs favourably in comparison with the widely used LSA approach, a Naive Bayes approach, and finally a purely symbolic approach.
Keywords. Tutorial dialogue, language understanding, evaluation
