Paper Info
Title
Multimodal registration of visible, SWIR and LWIR images in a distributed smart camera system
Abstract
We present a multimodal registration algorithm between images in the visible, short-wave infrared and long-wave infrared spectra. The algorithm works with two reference-objective image pairs and operates in two stages: (1) A calibration phase between static frames to estimate the transformation parameters using histogram of oriented gradients and the Chi-square distance; (2) a frame-by-frame mapping with these parameters using a projective transformation and a bilinear interpolation to map the objective video stream to the coordinate system of the reference video stream. We present a distributed heterogeneous architecture that combines a programmable processor core and a custom hardware accelerator for each node. The software performs the calibration phase, whereas the hardware computes the frame-by-frame mapping. We implemented our design using a Xilinx Zynq XC7Z020 system-on-a-chip for each node. The prototype uses 2.38W of power, 25% of the logic resources and 65% of the available on-chip memory per node. Running at 100MHz, the core can register 640  ×  512-pixel frames in 4ms after initial calibration, which allows our module to operate at up to 250 frames per second.
Year
DOI
Venue
2020
10.1016/j.micpro.2019.102987
Microprocessors and Microsystems
Keywords
Field
DocType
Short-wave infrared,Long-wave infrared,Image registration,Embedded systems,Video processing,FPGA
Coordinate system,Computer vision,Computer science,Smart camera,Real-time computing,Software,Histogram of oriented gradients,Homography,Frame rate,Artificial intelligence,Multi-core processor,Bilinear interpolation
Journal
Volume
ISSN
Citations 
73
0141-9331
0
PageRank 
References 
Authors
0.34
0
3
Name
Order
Citations
PageRank
Javier Cardenas141.53
Javier E. Soto200.34
Miguel Figueroa325732.96