This presentation was made on a thesis paper for my M.Sc academic curriculum. Color Guided Thermal image Super Resolution Technic is declared here.This paper is collected from IEEE.Publish in 2016.
3. COLOR GUIDED THERMAL IMAGE SUPER
RESOLUTION
Xiaohui Chen(1) , Guangtao Zhai1 , Jia Wang (2) ,Chunjia Hu (1) and
Yuanchun Chen(2)
Institute of Image Communication and Information Processing, Shanghai Jiao Tong
University, Shanghai, China
• 1 {(fxhuichen.cn,zhaiguangtao,huchunjia.sjtug)@gmail.com},
2{(fjiawang,chenyuanchung)@sjtu.edu.cn}
4. ABSTRACT OF THIS PAPER
Use the visible camera as a guidance to supper resolution the IR
images.
Setup a prototype of the IR-color multi-sensor imaging system
Construct a dataset including videos collected from different scenes
for further research.
Present their color guided algorithm which is suitable for this kind of
super resolution problem.
5. INTRODUCTION
A registered high-quality texture image can provide significant information to
enhance the infrared image due to their strong correlation.
Use a guided filter[1] in the correlated region between IR image and color image.
Construct a cost volume of IR image values probability based on the input image.
A best cost selecting and sub-pixel refinement are taken to produce a refined IR
image.
The output image is got after a outlier detection.
1. Z. Zhang, “A flexible new technique for camera calibration,” Pattern
6. SYSTEM SETUP AND PREPROCESSING
System Setting
Camera Calibration and Color- IR Registration
Proposed Approach
8. CAMERA CALIBRATION AND COLOR-IR
REGISTRATION
First they tried to use mature algorithm proposed by Zhang [1].
Then finally they used the method proposed by Han [2].
1. Z. Zhang, “A flexible new technique for camera calibration,” Pattern
2. J. Han, E. Pauwels, and P. de Zeeuw, “Visible and infrared image registration employing line-based geometric analysis,” in Computational Intelligence for Multimedia
Understanding. Springer, 2011, pp. 114–125.
11. COST VOLUME
A coarse cost volume is first built to preserve the sub-pixel accuracy of the input
IR image based on current IR image estimation Ii
As the differences get large, the cost function should become constant in order to
make candidate IR values vary a lot because the current values are not
necessarily correct. The truncated quadric model is one of available function.
Thus the cost function can be defined as following:
Where L is the search range,ƞ is a constant, d means a candidate IR image.
12. GUIDANCE SELECTION
The infrared image discontinuities often co-occur with color or
brightness changes within the associated camera image of the
same scene.
A color camera is a combination of three sensors: red, green, and
blue. Different channels have different correlation with IR image. To
verify this, we calculate the PSNR and SSIM between different
channels and the corresponding IR images in our database.
14. THE CORRELATED REGION
To avoid wrong texture transfer, they only applied guided
filter in their correlated region. The correlated region could
be calculated by their cross-correlation in a small patch. If
the value is bigger than the threshold T, the point is regard
as one point of correlated region. The others is in
uncorrelated region.
15. GUIDED FILTER IN CORRELATED REGION
The guided image filter is a novel explicit image filter. It
not only has good edge-preserving smoothing properties
like the bilateral filter[1], but also makes the filtering output
more structured and keeps more details than other edge-
preserving filters.
1. C. Tomasi and R. Manduchi, “Bilateral filtering for gray and color
16. SUB-PIXEL ESTIMATION
As we mentioned in Cost volume, the discontinuity of cost
function and limited search range result in the
discontinuity of IR images. In order to eliminate this effect,
quadratic polynomial interpolation is adopted here to
approximate the cost function.
17. OUTLIER DETECTION
Outlier detection is used here to solve the problem that sometimes
black points may appear where tiny edges only exist in IR image.
we use outlier detection method to find the incorrect points and then
we adopt a median filter around them to eliminate outlier points.
19. EXPERIMENTAL RESULTS
Experiment results. From left to right are the suitable channel images, the ground truth IR images, the bicubic
interpolation results, the result using JBU and the results using our proposed algorithm. The result images has
been upsampled by a factor of 1:4 (in each axis).
21. ACKNOWLEDGEMENT
• This work was supported in part by the National Science Foundation of China
under Grants 61422112, 61371146, 61521062, 61527804, and National High-tech
R&D Program of China under Grant 2015AA01590.