Paper Info
Title
Enhancing Mobile Voice Assistants with WorldGaze
Abstract
Contemporary voice assistants require that objects of inter-est be specified in spoken commands. Of course, users are often looking directly at the object or place of interest ? fine-grained, contextual information that is currently unused. We present WorldGaze, a software-only method for smartphones that provides the real-world gaze location of a user that voice agents can utilize for rapid, natural, and precise interactions. We achieve this by simultaneously opening the front and rear cameras of a smartphone. The front-facing camera is used to track the head in 3D, including estimating its direction vector. As the geometry of the front and back cameras are fixed and known, we can raycast the head vector into the 3D world scene as captured by the rear-facing camera. This allows the user to intuitively define an object or region of interest using their head gaze. We started our investigations with a qualitative exploration of competing methods, before developing a functional, real-time implementation. We conclude with an evaluation that shows WorldGaze can be quick and accurate, opening new multimodal gaze+voice interactions for mobile voice agents.
Year
DOI
Venue
2020
10.1145/3313831.3376479
CHI '20: CHI Conference on Human Factors in Computing Systems Honolulu HI USA April, 2020
DocType
ISBN
Citations 
Conference
978-1-4503-6708-0
0
PageRank 
References 
Authors
0.34
0
3
Name
Order
Citations
PageRank
Sven Mayer118827.30
Gierad Laput247125.01
Chris Harrison32800150.70