Title
An Interactive Mixed Reality Framework for Virtual Humans
Abstract
In this paper, we present a simple and robust Mixed Reality (MR) framework that allows for real-time interaction with Virtual Humans in real and virtual environments under consistent illumination. We will look at three crucial parts of this system: interaction, animation and global illumination of virtual humans for an integrated and enhanced presence. The interaction system comprises of a dialogue module, which is interfaced with a speech recognition and synthesis system. Next to speech output, the dialogue system generates face and body motions, which are in turn managed by the virtual human animation layer. Our fast animation engine can handle various types of motions, such as normal key-frame animations, or motions that are generated on-the-fly by adapting previously recorded clips. All these different motions are generated and blended on-line, resulting in a flexible and realistic animation. Our robust rendering method operates in accordance with the previous animation layer, based on an extended for virtual humans Precomputed Radiance Transfer (PRT) illumination model, resulting in a realistic display of such interactive virtual characters in mixed reality environments. Finally, we present a scenario that illustrates the interplay and application of our methods, glued under a unique framework for presence and interaction in MR.
Year
DOI
Venue
2006
10.1109/CW.2006.15
CW
Keywords
Field
DocType
virtual human,interactive mixed reality,previous animation layer,interaction system,fast animation engine,normal key-frame animation,dialogue system,virtual human animation layer,interactive virtual character,virtual environment,realistic animation,virtual humans,mixed reality,speech recognition,speech synthesis,computer animation,real time
Virtual machine,Interactive skeleton-driven simulation,Computer graphics (images),Computer science,Artificial intelligence,Virtual actor,Computer vision,Animation,Mixed reality,Precomputed Radiance Transfer,Rendering (computer graphics),Computer animation,Multimedia
Conference
ISBN
Citations 
PageRank 
0-7695-2671-3
6
0.74
References 
Authors
19
3
Name
Order
Citations
PageRank
Arjan Egges141732.06
George Papagiannakis223422.95
Nadia Magnenat-Thalmann35119659.15