Paper Info
Title
A Matrix Factorization Based Framework For Fusion Of Physical And Social Sensors
Abstract
Our world is witnessing the on-going substantial increase in the number of multimodal physical and social sensors that are ubiquitously distributed and observing or reporting what is happening in their surroundings. These sensors provide massive amounts of spatio-temporal digital footprints which can be analyzed for various tasks such as event detection or situation awareness. However, inherent noise due to the nature of these sensors result in imprecise data and hence imprecise analysis. Also, the heterogeneous data from different modalities, formats and sources make interpreting different levels of information a big challenge. To overcome these limitations, we propose a novel unified matrix factorization-based model to fuse physical and social sensor signals for spatio-temporal analysis. Readings of physical sensor signals are represented by a spatio-temporal situation matrix, which then incorporates social content that can provide explanations for the signal strengths. We test our framework on large-scale real-world data including PSI stations data, traffic CCTV camera images, and tweets for situation prediction as well as for filtering noise to detect events of diverse situations. The experimental results suggest that the proposed matrix factorization approach can utilize the sources correlation, resulting in better performances in various situational understanding tasks.
Year
DOI
Venue
2021
10.1109/TMM.2020.3016222
IEEE TRANSACTIONS ON MULTIMEDIA
Keywords
DocType
Volume
Sensor fusion, Task analysis, Cameras, Semantics, Event detection, Image sensors, Physical social fusion, multimodal and multisource analysis, situation awareness, spatial temporal filtering
Journal
23
ISSN
Citations 
PageRank 
1520-9210
0
0.34
References 
Authors
0
4
Name
Order
Citations
PageRank
Yuhui Wang100.34
Francesco Gelli2544.02
Christian von der Weth333.42
Mohan Kankanhalli43825299.56