This document summarizes work on video super-resolution techniques. It introduces two algorithms: FGSR, a fast general super-resolution method; and FRSR, a slower but higher quality method for translational motion. It evaluates the algorithms on test video, finding that FRSR achieves higher PSNR scores than FGSR or bilinear interpolation. Overall processing speed is improved with a GPU implementation. The conclusion discusses open challenges in optical flow, occlusion handling, and utilizing more frame information.

1. 薛健鑫, 蔡昆修, 何星翰
1

2.  Introduction
 Work progress
 SR Algorithm
 Evaluation
 SR video player demo
 Conclusion
2

3.  Super-resolution are techniques that improve
image quality from low-resolution.
3

4.  Multiple neighboring frames of video each provide a
rich amount of information about scene details.
 Super-resolution on video can analyze multiple
frames to reconstruct a enhance frame with detail
4

5.  Retrieving image stream
 From OpenCV
 Super-resolution processing
 Algorithm : FGSR, FRSR
 GPU speedup investigation
 Video player
5

6.  Fast General Super-Resolution (FGSR)
 For general video use
 Fast, cheap, bad result
 Fast & Robust Super-Resolution(FRSR)
 Only appropriate for translational motion
 Slow, better result
6

7.  Predict HR = bicubic( Gi )
 Iterative fix Predict HR :
In’ = optcial_flow( bicubic(Gi+n ) )
Predict HR += α * ( Predict HR – In’ )
*
7

8. Processing time
500
450
400
350
300
time (ms)
250
cpu
200 gpu
150
100
50
0
64 128 256 512 1024 2048
Image size
8

9. Bilinear interpolation FGSR
9

10.  Predicted HR= Median( shift(LR0~n) )
 Iterative improve Predicted HR :
Gback = FastGradientBackProject(HR);
Greg = GradientRegularization(HR);
HR = HR - β*(Gback + α* Greg);
10

11. Bilinear interpolation FRSR
11

12.  PSNR (Peak Signal to Noise Ratio):
Ground truth Bilinear interpolation Bicubic
interpolation
image
PSNR ∞ 14.450087 15.79615
(dB)
12

13.  PSNR (Peak Signal to Noise Ratio):
Ground truth FGSR FRSR
algorithm algorithm
image
PSNR ∞ 14.642818 15.150817
(dB)
13

14. 14

15.  Problem on super-resolution:
 Optical flow
 Occlusion/ disocclusion
 Insufficient information from neighboring frames
 Still have much space to improve
15

16.  Sina Farsiu, Dirk Robinson, Michael Elad, Peyman Milanfar.
Fast and Robust Multi-Frame Super-Resolution. In IEEE
Transactions on Image Processing2003.
 Zhongding Jiang, Tien-tsin Wong, Hujun Bao. Practical
Super-Resolution from Dynamic Video Sequences. In Proc. of
IEEE CVPR2003.
16

17. THE END
17