Title
Seeing is believing: body motion dominates in multisensory conversations
Abstract
In many scenes with human characters, interacting groups are an important factor for maintaining a sense of realism. However, little is known about what makes these characters appear realistic. In this paper, we investigate human sensitivity to audio mismatches (i.e., when individuals' voices are not matched to their gestures) and visual desynchronization (i.e., when the body motions of the individuals in a group are mis-aligned in time) in virtual human conversers. Using motion capture data from a range of both polite conversations and arguments, we conduct a series of perceptual experiments and determine some factors that contribute to the plausibility of virtual conversing groups. We found that participants are more sensitive to visual desynchronization of body motions, than to mismatches between the characters' gestures and their voices. Furthermore, synthetic conversations can appear sufficiently realistic once there is an appropriate balance between talker and listener roles. This is regardless of body motion desynchronization or mismatched audio.
Year
DOI
Venue
2010
10.1145/1833349.1778828
ACM Transactions on Graphics
Keywords
Field
DocType
multisensory conversation,virtual human conversers,virtual conversing group,human sensitivity,visual desynchronization,body motion desynchronization,human character,motion capture data,body motion,mismatched audio,audio mismatches,conversational agent,computer graphics,perception,virtual human
Computer vision,Motion capture,Crowds,Computer graphics (images),Gesture,Computer science,Politeness,Artificial intelligence,Virtual actor,Perception
Journal
Volume
Issue
ISSN
29
4
0730-0301
Citations 
PageRank 
References 
19
1.11
10
Authors
3
Name
Order
Citations
PageRank
Cathy Ennis11278.74
Rachel McDonnell255849.37
Carol O'Sullivan382548.93