Paper Info
Title
Spontaneous smile intensity estimation by fusing saliency maps and convolutional neural networks.
Abstract
Smile intensity estimation plays important roles in applications such as affective disorder prediction, life satisfaction prediction, camera technique improvement, etc. In recent studies, many researchers applied only traditional features, such as local binary pattern and local phase quantization (LPQ) to represent smile intensity. To improve the performance of spontaneous smile intensity estimation, we introduce a feature set that combines the saliency map (SM)-based handcrafted feature and non-low-level convolutional neural network (CNN) features. We took advantage of the opponent-color characteristic of SMs and the multiple convolutional level features, which were assumed to be mutually complementary. Experiments were made on the Binghamton-Pittsburgh 4D (BP4D) database and Denver Intensity of Spontaneous Facial Action (DISFA) database. We set the local binary patterns on three orthogonal planes (LBPTOP) method as a baseline, and the experimental results show that the CNN features can better estimate smile intensity. Finally, through the proposed SM-LBPTOP feature fusion with the median- and high-level CNN features, we obtained the best result (52.08% on BP4D, 70.55% on DISFA), demonstrating our hypothesis is reasonable: the SM-based handcrafted feature is a good supplement to CNNs in spontaneous smile intensity estimation. (C) 2019 SPIE and IS&T
Year
DOI
Venue
2019
10.1117/1.JEI.28.2.023031
JOURNAL OF ELECTRONIC IMAGING
Keywords
Field
DocType
smile intensity,saliency maps,convolutional neural network
Computer vision,Pattern recognition,Convolutional neural network,Computer science,Salience (neuroscience),Artificial intelligence
Journal
Volume
Issue
ISSN
28
2
1017-9909
Citations 
PageRank 
References 
0
0.34
0
Authors
4
Name
Order
Citations
PageRank
Qinglan Wei1272.44
Elif Bozkurt2636.65
Louis-Philippe Morency33220200.79
Bo Sun410421.35