Paper Info
Title
Improving Classification of Natural Language Answers to ITS Questions with Item-Specific Supervised Learning
Abstract
In a natural language intelligent tutoring system, improving assessment of student input is a challenge that typically requires close collaboration between domain experts and NLP experts. This paper proposes a method for building small item-specific classifiers that would allow a domain expert author to improve quality of assessment for student input, through supervised tagging of a small number of good and bad examples. Our approach then generates large sets of item-specific features, which are reduced by algorithms (e.g., Support Vector Machines, Ridge Regression) designed for high-dimensional feature sets. The feasibility of this approach is evaluated using an existing data set collected from an intelligent tutoring system based on AutoTutor. Evaluations of this technique show that it performs about as effectively as iteratively hand-tuned regular expressions for evaluating student input, which is a time-consuming process. This method also outperforms general Latent Semantic Analysis for evaluating the quality of input after approximately 8 tagged examples are available. By 16 tagged examples, it improves the correlation with human raters to over R=0.50, as compared to about R=0.25 for LSA alone. As such, this approach could be used by ITS authors to improve the quality of natural language understanding for their content by classifying a fairly small number of student inputs. Future directions for this work include integrating this approach into an authoring system and exploring directions for decreasing the number of tagged examples that are needed to provide effective classifications.
Year
Venue
DocType
2015
FLAIRS Conference
Conference
Citations 
PageRank 
References 
0
0.34
6
Authors
3
Name
Order
Citations
PageRank
Benjamin D. Nye15710.62
Mustafa H. Hajeer262.49
zhiqiang cai334478.81