To investigate whether more concise Natural Language feedback improves learning, we developed
two Natural Language generators (DIAG-NLP1 and DIAG-NLP2), to provide feedback in an Intelligent Tutoring
System that teaches troubleshooting. We systematically evaluated them in a three way comparison that included
the original system, which generates overly repetitive feedback. We found that DIAG-NLP2, the generator which
intuitively produces the best, corpus-based language, does engender the most learning. Distinguishing features of
the more effective feedback are: it obeys Grice's maxim of brevity, it is more directive and uses a specific type
of referring expressions. Interestingly, simpler ways of restructuring the original repetitive feedback as done in
DIAG-NLP1, such as exploiting the hierarchical structure of the domain, were not effective. Since the design
of interfaces to Intelligent Tutoring Systems often includes verbal feedback, we suggest that: if the number of
different contexts in which verbal feedback is provided is high, such feedback should be based on corpus studies,
and generated by techniques more sophisticated than template filling.
