Paper Info
Title
Learning and synthesizing MPEG-4 compatible 3-D face animation from video sequence
Abstract
We present a new system that applies an example-based learning method to learn facial motion patterns from a video sequence of individual facial behavior such as lip motion and facial expressions, and using that to create vivid three-dimensional (3-D) face animation according to the definition of MPEG-4 face animation parameters. The system consists of three key modules, face tracking, pattern learning, and face animation. In face tracking, to reduce the complexity of the tracking process, a novel coarse-to-fine strategy combined with a Kalman filter is proposed for localizing key facial landmarks in each image of the video. The landmarks' sequence is normalized into a visual feature matrix and then fed to the next step of process. In pattern learning, in the pretraining stage, the parameters of the camera that took the video are requested with the training video data so the system can estimate the basic mapping from a normalized two-dimensional (2-D) visual feature matrix to the representation in 3-D MPEG-4 face animation parameter space, in assistance with the computer vision method. In the practice stage, considering that in most cases camera parameters are not provided with video data, the system uses machine learning technology to complement the incomplete 3-D information for the mapping that information is needed in face orientation presentation. The example-based learning in this system integrates several methods including clustering, HMM, and ANN to make a better conversion from a 2-D to 3-D model and better estimation of incomplete 3-D information for good mapping; this will be used to drive face animation thereafter. In face animation, the system can synthesize face animation following any type of face motion in video. Experiments show that our system produces more vivid face motion animation, compared to other early systems.
Year
DOI
Venue
2003
10.1109/TCSVT.2003.817629
IEEE Trans. Circuits Syst. Video Techn.
Keywords
Field
DocType
face tracking,incomplete 3-d information,early system,mpeg-4 face animation parameter,mpeg-4.,face animation,face orientation presentation,pattern learning,vivid face motion animation,face motion,index terms—face animation,visual feature matrix,machine learning,mpeg-4 compatible 3-d face,video sequence,kalman filter,hidden markov models,computer vision,three dimensional,facial expression,data compression,kalman filters,system integration,computer animation,parameter space,indexing terms,learning artificial intelligence,feature extraction,neural nets
Computer vision,Pattern recognition,Computer science,Feature extraction,Artificial intelligence,Computer facial animation,Skeletal animation,Animation,Face detection,Computer animation,Face Animation Parameter,Facial motion capture
Journal
Volume
Issue
ISSN
13
11
1051-8215
Citations 
PageRank 
References 
17
0.75
22
Authors
5
Name
Order
Citations
PageRank
Wen Gao111374741.77
Yiqiang Chen21446109.32
Rui Wang3302.19
Shiguang Shan46322283.75
Dalong Jiang520310.26