Title
The Good, The Bad, And The Ugly: Neural Networks Straight From Jpeg
Abstract
Over the past decade, convolutional neural networks (CNNs) have achieved state-of-the-art performance in many computer vision tasks. They can learn robust representations of image data by processing RGB pixels. Since image data are often stored in a compressed format, from which JPEG is the most widespread, a preliminary decoding process is demanded. Recently, the design of CNNs for processing JPEG compressed data has gained attention from the research community. They process DCT coefficients instead of RGB pixels, saving computation for decoding JPEG images, however, at the cost of increasing the computational complexity of the network. In this paper, we examine how spatial resolution and JPEG quality impacts on the performance of a state-of-the-art CNN designed to operate directly on the JPEG compressed domain. To alleviate its computational complexity, we propose a Frequency Band Selection (FBS) technique to select the most relevant DCT coefficients before feeding them to the network. Experiments were conducted on a subset of the ImageNet dataset considering both fine- and coarse-grained image classification tasks. Results show that such networks are resilient to JPEG quality but are susceptible to spatial resolution. Also, our FBS can reduce the computational complexity of the network while retaining a similar accuracy.
Year
DOI
Venue
2020
10.1109/ICIP40778.2020.9190741
2020 IEEE INTERNATIONAL CONFERENCE ON IMAGE PROCESSING (ICIP)
Keywords
DocType
ISSN
Deep Learning, Convolutional Neural Network, Compressed-Domain Processing, JPEG
Conference
1522-4880
Citations 
PageRank 
References 
2
0.38
0
Authors
3
Name
Order
Citations
PageRank
Samuel Felipe dos Santos120.38
Nicu Sebe27013403.03
Jurandy Almeida343135.15