Name
Title | ||
|---|---|---|
Automatic inference of cross-modal nonverbal interactions in multiparty conversations: "who responds to whom, when, and how?" from gaze, head gestures, and utterances |
Abstract | ||
|---|---|---|
A novel probabilistic framework is proposed for analyzing cross-modal nonverbal interactions in multiparty face-to-face conversations. The goal is to determine "who responds to whom, when, and how" from multimodal cues including gaze, head gestures, and utterances. We formulate this problem as the probabilistic inference of the causal relationship among participants' behaviors involving head gestures and utterances. To solve this problem, this paper proposes a hierarchical probabilistic model; the structures of interactions are probabilistically determined from high-level conversation regimes (such as monologue or dialogue) and gaze directions. Based on the model, the interaction structures, gaze, and conversation regimes, are simultaneously inferred from observed head motion and utterances, using a Markov chain Monte Carlo method. The head gestures, including nodding, shaking and tilt, are recognized with a novel Wavelet-based technique from magnetic sensor signals. The utterances are detected using data captured by lapel microphones. Experiments on four-person conversations confirm the effectiveness of the framework in discovering interactions such as question-and-answer and addressing behavior followed by back-channel responses. |
Year | DOI | Venue |
|---|---|---|
2007 | 10.1145/1322192.1322237 | ICMI |
Keywords | Field | DocType |
automatic inference,high-level conversation regime,four-person conversation,hierarchical probabilistic model,conversation regime,multiparty conversation,observed head motion,cross-modal nonverbal interaction,novel wavelet-based technique,probabilistic inference,multiparty face-to-face conversation,head gesture,novel probabilistic framework,bayesian network,eye gaze,probabilistic model,gibbs sampler,data capture,markov chain monte carlo | Computer vision,Conversation,Markov chain Monte Carlo,Gaze,Gesture,Computer science,Nonverbal communication,Speech recognition,Eye tracking,Bayesian network,Statistical model,Artificial intelligence | Conference |
Citations | PageRank | References |
43 | 1.85 | 11 |
Authors | ||
3 | ||
Name | Order | Citations | PageRank |
|---|---|---|---|
| Kazuhiro Otsuka | 1 | 619 | 54.15 |
| Hiroshi Sawada | 2 | 1809 | 136.96 |
| Junji Yamato | 3 | 1120 | 165.72 |