Name
Title | ||
|---|---|---|
Towards Automated Understanding of Student-Tutor Interactions Using Visual Deictic Gestures |
Abstract | ||
|---|---|---|
In this paper, we present techniques for automated understanding of tutor-student behavior through detecting visual deictic gestures, in the context of one-to-one mathematics tutoring. To the best knowledge of the authors, this is the first work in the area of intelligent tutoring systems, which focuses on spatial localization of deictic gestural activity, i.e. where the deictic gesture is pointing on the workspace. A new dataset called SDMATH is first introduced. The motivation for detecting deictic gestures and their spatial properties is established, followed by techniques for automatic localization of deictic gestures in a workspace. The techniques employ computer vision and machine learning steps such as GBVS saliency, binary morphology and HOG-SVM classification. It is shown that the method localizes the deictic tip with an accuracy of over 85 % accuracy for a cut off distance of 12 pixels. Furthermore, a detailed discussion using examples from the proposed dataset is presented on high-level inferences about the student-tutor interactions that can be derived from the integration of spatial and temporal localization of the deictic gestural activity using the proposed techniques. |
Year | DOI | Venue |
|---|---|---|
2014 | 10.1109/CVPRW.2014.77 | CVPR Workshops |
Keywords | Field | DocType |
sdmath,intelligent tutoring systems,student-tutor interaction automated understanding,learning (artificial intelligence),gbvs saliency,deictic gesture automatic localization,high-level inferences,hog-svm classification,mathematics computing,image classification,object detection,computer vision,binary morphology,gesture recognition,machine learning,one-to-one mathematic tutoring,temporal localization integration,spatial localization integration,deictic gestural activity spatial localization,support vector machines,visual deictic gesture detection,learning artificial intelligence,mathematics,visualization,speech,bismuth | Computer vision,TUTOR,Workspace,Computer science,Salience (neuroscience),Gesture,Spatial localization,Natural language processing,Pixel,Artificial intelligence,Deixis | Conference |
ISSN | Citations | PageRank |
2160-7508 | 2 | 0.36 |
References | Authors | |
13 | 9 | |
Name | Order | Citations | PageRank |
|---|---|---|---|
| Suchitra Sathyanarayana | 1 | 20 | 4.36 |
| Ravi Kumar Satzoda | 2 | 77 | 10.07 |
| Amber R. Carini | 3 | 4 | 0.80 |
| Monique A. Lee | 4 | 4 | 0.80 |
| Linda P. Salamanca | 5 | 26 | 2.37 |
| Judy Reilly | 6 | 2 | 0.36 |
| Deborah Forster | 7 | 24 | 3.93 |
| Marian Stewart Bartlett | 8 | 2026 | 183.92 |
| gwen littlewort | 9 | 1159 | 67.40 |