Paper Info
Title
Detecting Gaze Towards Eyes in Natural Social Interactions and Its Use in Child Assessment
Abstract
Eye contact is a crucial element of non-verbal communication that signifies interest, attention, and participation in social interactions. As a result, measures of eye contact arise in a variety of applications such as the assessment of the social communication skills of children at risk for developmental disorders such as autism, or the analysis of turn-taking and social roles during group meetings. However, the automated measurement of visual attention during naturalistic social interactions is challenging due to the difficulty of estimating a subject’s looking direction from video. This paper proposes a novel approach to eye contact detection during adult-child social interactions in which the adult wears a point-of-view camera which captures an egocentric view of the child’s behavior. By analyzing the child’s face regions and inferring their head pose we can accurately identify the onset and duration of the child’s looks to their social partner’s eyes. We introduce the Pose-Implicit CNN, a novel deep learning architecture that predicts eye contact while implicitly estimating the head pose. We present a fully automated system for eye contact detection that solves the sub-problems of end-to-end feature learning and pose estimation using deep neural networks. To train our models, we use a dataset comprising 22 hours of 156 play session videos from over 100 children, half of whom are diagnosed with Autism Spectrum Disorder. We report an overall precision of 0.76, recall of 0.80, and an area under the precision-recall curve of 0.79, all of which are significant improvements over existing methods.
Year
DOI
Venue
2019
10.1145/3131902
Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies
Keywords
DocType
Volume
Wearable camera,assessment,autism spectrum disorder,computer vision,deep learning,eye contact,gaze classification,machine learning
Journal
1
Issue
ISSN
Citations 
3
2474-9567
7
PageRank 
References 
Authors
0.62
20
8
Name
Order
Citations
PageRank
Eunji Chong1152.89
Katha Chanda270.62
Zhefan Ye31014.79
Audrey Southerland470.62
Nataniel Ruiz5163.25
Rebecca M Jones6322.84
Agata Rozga714912.11
James M. Rehg85259474.66