Paper Info
Title
Decoding Auditory Saliency from FMRI Brain Imaging
Abstract
Given the growing number of available audio streams through a variety of sources and distribution channels, effective and advanced computational audio analysis has received increasing interest in the multimedia field. However, the effectiveness of current audio analysis strategies might be hampered due to the lack of effective representation of high-level semantics perceived by the human and the lack of effective approaches to bridging the gaps between most low-level acoustic features and high-level semantic features. This semantic gap has become the 'bottleneck' problem in audio analysis. In this paper, we propose a computational framework to decode biologically-plausible auditory saliency using high-level features derived from functional magnetic resonance imaging (fMRI) which monitors the human brain's response under the natural stimulus of audio listening. Specifically, we identify meaningful intrinsic brain networks which are involved in audio listening via effective online dictionary learning and sparse representation of whole-brain fMRI signals, reconstruct auditory saliency features using those identified brain network components, and perform group-wise analysis to identify consistent 'brain decoders' of the saliency features across different excerpts and participants. Experimental results demonstrate that the auditory saliency features are effectively decoded via our methods, which potentially provide opportunities for various applications in the multimedia field.
Year
DOI
Venue
2014
10.1145/2647868.2655039
ACM Multimedia 2001
Keywords
Field
DocType
algorithms,design,experimentation,human factors,human information processing,multimedia information systems
Computer vision,Functional magnetic resonance imaging,Computer science,Salience (neuroscience),Semantic gap,Sparse approximation,Active listening,Speech recognition,Audio analyzer,Artificial intelligence,Neuroimaging,Semantics
Conference
Citations 
PageRank 
References 
1
0.35
9
Authors
9
Name
Order
Citations
PageRank
Shijie Zhao171.50
Xi Jiang262.11
Junwei Han33501194.57
Xintao Hu451.75
Dajiang Zhu532036.72
Jinglei Lv620526.70
Tuo Zhang774.47
Lei Guo81661142.63
Tianming Liu91033112.95