Name
Abstract | ||
|---|---|---|
Facial action units (AU) are the fundamental units to decode human facial expressions. At least three aspects affect performance of automated AU detection: spatial representation, temporal modeling, and AU correlation. Unlike most studies that tackle these aspects separately, we propose a hybrid network architecture to jointly model them. Specifically, spatial representations are extracted by a Convolutional Neural Network (CNN), which, as analyzed in this paper, is able to reduce person-specific biases caused by hand-crafted descriptors (e.g., HOG and Gabor). To model temporal dependencies, Long Short-Term Memory (LSTMs) are stacked on top of these representations, regardless of the lengths of input videos. The outputs of CNNs and LSTMs are further aggregated into a fusion network to produce per-frame prediction of 12 AUs. Our network naturally addresses the three issues together, and yields superior performance compared to existing methods that consider these issues independently. Extensive experiments were conducted on two large spontaneous datasets, GFT and BP4D, with more than 400,000 frames coded with 12 AUs. On both datasets, we report improvements over a standard multi-label CNN and feature-based state-of-the-art. Finally, we provide visualization of the learned AU models, which, to our best knowledge, reveal how machines see AUs for the first time. |
Year | DOI | Venue |
|---|---|---|
2017 | 10.1109/FG.2017.13 | 2017 12th IEEE International Conference on Automatic Face & Gesture Recognition (FG 2017) |
Keywords | Field | DocType |
temporal cues learning,spatial cues learning,multilabel facial action unit detection,human facial expression decoding,automated AU detection,spatial representation,temporal modeling,AU correlation,hybrid network architecture,spatial representations,convolutional neural network,CNN,person-specific biases reduction,hand-crafted descriptors,temporal dependency modelling,long short-term memory,LSTMs,fusion network | Scale-invariant feature transform,Pattern recognition,Visualization,Convolutional neural network,Computer science,Network architecture,Facial expression,Correlation,Artificial intelligence,Spatial representation,Temporal modeling,Machine learning | Conference |
ISSN | ISBN | Citations |
2326-5396 | 978-1-5090-4024-7 | 9 |
PageRank | References | Authors |
0.50 | 0 | 3 |
Name | Order | Citations | PageRank |
|---|---|---|---|
| Wen-Sheng Chu | 1 | 380 | 14.54 |
| Fernando De La Torre | 2 | 3832 | 181.17 |
| Jeffrey F. Cohn | 3 | 5438 | 343.74 |