Title
Multi-Modal Object Tracking and Image Fusion With Unsupervised Deep Learning
Abstract
The number of different modalities for remote sensors continues to grow, bringing with it an increase in the volume and complexity of the data being collected. Although these datasets individually provide valuable information, in aggregate they provide additional opportunities to discover meaningful patterns on a large scale. However, the ability to combine and analyze disparate datasets is challenged by the potentially vast parameter space that results from aggregation. Each dataset in itself requires instrument-specific and dataset-specific knowledge. If the intention is to use multiple, diverse datasets, one needs an understanding of how to translate and combine these parameters in an efficient and effective manner. While there are established techniques for combining datasets from specific domains or platforms, there is no generic, automated method that can address the problem in general. Here, we discuss the application of deep learning to track objects across different image-like data-modalities, given data in a similar spatio-temporal range, and automatically co-register these images. Using deep belief networks combined with unsupervised learning methods, we are able to recognize and separate different objects within image-like data in a structured manner, thus making progress toward the ultimate goal of a generic tracking and fusion pipeline requiring minimal human intervention.
Year
DOI
Venue
2019
10.1109/JSTARS.2019.2920234
IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing
Keywords
Field
DocType
Instruments,Remote sensing,Earth,Data models,Deep learning,Cameras,Data integration
Data integration,Computer vision,Data modeling,Image fusion,Deep belief network,Unsupervised learning,Video tracking,Artificial intelligence,Deep learning,Machine learning,Modal,Mathematics
Journal
Volume
Issue
ISSN
12
8
1939-1404
Citations 
PageRank 
References 
0
0.34
0
Authors
5
Name
Order
Citations
PageRank
Nicholas LaHaye111.38
Jordan Ott2111.32
Michael J. Garay3138.54
Hesham Mohamed El-Askary400.68
Erik Linstead536027.44