Paper Info
Title
Semi-Supervised Deep Neural Network for Joint Intensity Estimation of Multiple Facial Action Units
Abstract
Facial action units (AUs) are defined to depict movements of facial muscles, which are basic elements to encode facial expressions. Automatic AU intensity estimation is an important task in affective computing. Previous works leverage the representation power of deep neural networks (DNNs) to improve the performance of intensity estimation. However, a large number of intensity annotations are required to train DNNs that contain millions of parameters. But it is expensive and difficult to build a large-scale database with AU intensity annotation since AU annotation requires annotators have strong domain expertise. We propose a novel semi-supervised deep convolutional network that leverages extremely limited AU annotations for AU intensity estimation. It requires only intensity annotations of keyframes of training sequences. Domain knowledge on AUs is leveraged to provide weak supervisory information, including relative appearance similarity, temporal intensity ordering, facial symmetry, and contrastive appearance difference. We also propose a strategy to train a model for joint intensity estimation of multiple AUs under the setting of semi-supervised learning, which greatly improves the efficiency during inference. We perform empirical experiments on two public benchmark expression databases and make comparisons with state-of-the-art methods to demonstrate the effectiveness of the proposed method.
Year
DOI
Venue
2019
10.1109/ACCESS.2019.2947201
IEEE ACCESS
Keywords
DocType
Volume
Gold,Estimation,Hidden Markov models,Training,Face,Task analysis,Neural networks,Facial action units,intensity estimation,deep learning,weakly supervised learning
Journal
7
ISSN
Citations 
PageRank 
2169-3536
0
0.34
References 
Authors
0
5
Name
Order
Citations
PageRank
Yong Zhang1163.97
Yanbo Fan2176.36
Weiming Dong334935.73
Hu Bao-Gang4138683.23
Qiang Ji52780168.90