Name
Abstract | ||
|---|---|---|
One of the challenges of speech-to-speech translation is to accurately preserve the paralinguistic information in the speaker’s message. Information about affect and emotional intent of a speaker are often carried in more than one modality. For this reason, the possibility of multimodal interaction with the system and the conversation partner may greatly increase the likelihood of a successful and gratifying communication process. In this work we explore the use of automatic facial expression analysis as an input annotation modality to transfer paralinguistic information at a symbolic level from input to output in speech-to-speech translation. To evaluate the feasibility of this approach, a prototype system, FEAST (facial expression-based affective speech translation) has been developed. FEAST classifies the emotional state of the user and uses it to render the translated output in an appropriate voice style, using expressive speech synthesis. |
Year | DOI | Venue |
|---|---|---|
2014 | https://doi.org/10.1007/s12193-013-0128-x | Journal on Multimodal User Interfaces |
Keywords | Field | DocType |
Expressive speech synthesis,Speech-to-speech translation,Gesture-driven multimodal interface,Affective computing | Multimodal interaction,Speech synthesis,Conversation,Paralanguage,Computer science,Speech recognition,Facial expression,Natural language processing,Artificial intelligence,Speech translation,Affective computing,Affect (psychology) | Journal |
Volume | Issue | ISSN |
8 | 1 | 1783-7677 |
Citations | PageRank | References |
8 | 0.57 | 14 |
Authors | ||
4 | ||
Name | Order | Citations | PageRank |
|---|---|---|---|
| Éva Székely | 1 | 19 | 4.96 |
| Ingmar Steiner | 2 | 67 | 12.25 |
| Zeeshan Ahmed | 3 | 13 | 2.46 |
| Julie Carson-Berndsen | 4 | 75 | 28.62 |