Paper Info
Title
Data-based analysis of speech and gesture: the Bielefeld Speech and Gesture Alignment corpus (SaGA) and its applications
Abstract
Communicating face-to-face, interlocutors frequently produce multimodal meaning packages consisting of speech and accompanying gestures. We discuss a systematically annotated speech and gesture corpus consisting of 25 route-and-landmark-description dialogues, the Bielefeld Speech and Gesture Alignment corpus (SaGA), collected in experimental face-to-face settings. We first describe the primary and secondary data of the corpus and its reliability assessment. Then we go into some of the projects carried out using SaGA demonstrating the wide range of its usability: on the empirical side, there is work on gesture typology, individual and contextual parameters influencing gesture production and gestures’ functions for dialogue structure. Speech-gesture interfaces have been established extending unification-based grammars. In addition, the development of a computational model of speech-gesture alignment and its implementation constitutes a research line we focus on.
Year
DOI
Venue
2013
10.1007/s12193-012-0106-8
J. Multimodal User Interfaces
Keywords
Field
DocType
Speech-and-gesture alignment,Multimodal data,Multimodal simulation,Multimodal dialogue,Iconic gesture
Rule-based machine translation,Computer science,Gesture,Unification,Usability,Gesture recognition,Typology,Human–computer interaction,Natural language processing,Artificial intelligence
Journal
Volume
Issue
ISSN
7
1-2
1783-7677
Citations 
PageRank 
References 
6
0.55
24
Authors
5
Name
Order
Citations
PageRank
Andy Lücking1436.29
Kirsten Bergmann219917.95
Florian Hahn3625.32
stefan kopp49314.14
Hannes Rieser5526.29