Title
A Case Study Empirical Comparison of Three Methods to Evaluate Tutorial Behaviors
Abstract
Researchers have used various methods to evaluate the fine-grained interactions of intelligent tutors with their students. We present a case study comparing three such methods on the same data set, logged by Project LISTEN's Reading Tutor from usage by 174 children in grades 2-4 (typically 7-10 years) over the course of the 2005-2006 school year. The Reading Tutor chooses randomly between two different types of reading practice. In assisted oral reading, the child reads aloud and the tutor helps. In "Word Swap," the tutor reads aloud and the child identifies misread words. One method we use here to evaluate reading practice is conventional analysis of randomized controlled trials (RCTs), where the outcome is performance on the same words when encountered again later. The second method is learning decomposition, which estimates the impact of each practice type as a parameter in an exponential learning curve. The third method is knowledge tracing, which estimates the impact of practice as a probability in a dynamic Bayes net. The comparison shows qualitative agreement among the three methods, which is evidence for their validity.
Year
DOI
Venue
2008
10.1007/978-3-540-69132-7_17
Intelligent Tutoring Systems
Keywords
Field
DocType
word swap,project listen,reading tutor,evaluate tutorial behaviors,oral reading,conventional analysis,practice type,different type,various method,case study empirical comparison,case study,intelligent tutor,randomized controlled trials,learning curve,randomized controlled trial
Empirical comparison,TUTOR,Computer science,Randomized controlled trial,Bayesian network,Artificial intelligence,Educational data mining,Machine learning
Conference
Volume
ISSN
Citations 
5091
0302-9743
3
PageRank 
References 
Authors
0.50
8
3
Name
Order
Citations
PageRank
Xiaonan Zhang181.85
Jack Mostow21133263.51
Joseph E. Beck375989.98