This document summarizes a presentation on single image super resolution using fuzzy deep convolutional networks. It introduces the problem of super resolution and conventional approaches like manifold learning and dictionary learning. It then presents a proposed approach using a fuzzy deep convolutional network that incorporates a fuzzy rule layer into a convolutional neural network structure. This allows for task-driven feature learning while preserving spatial coherence. Experimental results show the proposed approach achieves better quantitative measures of PSNR, SSIM, and FSIM compared to methods like bicubic interpolation and SRCNN for magnification factors of 3. The findings conclude the method can better preserve structural information in the high resolution image with better visual quality while avoiding additional overhead during learning.

1. 18 January 2018 IEEE -Tap Energy 2017 1
Single Image Super Resolution using Fuzzy
Deep Convolutional Networks
Presented By
Greeshma M.S
School of Computer Sciences Mahatma Gandhi University,
Kottayam, Kerala, India
Authors
Greeshma M S, Dr. Bindu V R
Paper Id : 145
December 22, 2017
International (biennial) Conference on Technological Advancements in Power & Energy– TAP Energy 2017, Amrita University

2. 18 January 2018 IEEE -Tap Energy 2017 2
Introduction to super resolution problem
conventional Approaches
Single Image Super Resolution using Fuzzy Deep Convolutional
Networks
Experimental Results
Conclusion
“Want to Make Image Big &
Sharp”
Image Super –Resolution (SR)
Given a low resolution image , how do we make it larger without losing any details?
Low-Resolution
High-Resolution
SUPER
RESOLUTION
Applications
• Satellite Imaging
• Microscopy
• HD video generation from low-
resolution source
• Medical Imaging
• Object/ Text recognition from
Surveillance video
• ……………….

3. 18 January 2018 IEEE -Tap Energy 2017 3
Introduction to super resolution problem
conventional Approaches
Single Image Super Resolution using Fuzzy Deep Convolutional
Networks
Experimental Results
Conclusion
The “Playground”
Single Image Super Resolution (SISR)
Single Image
Super Resolution
Algorithm
Scale Factor
Image
Model
Manifold Learning Dictionary Learning Deep Learning
Very Challenging !
Are the generated details real?

4. 18 January 2018 IEEE -Tap Energy 2017 4
Introduction to super resolution problem
conventional Approaches
Single Image Super Resolution using Fuzzy Deep Convolutional
Networks
Experimental Results
Conclusion
The “Playground”
Problem Domain
Well designed efficient algorithm for increasing image resolution, maintaining better visual quality
and preserving information using Fuzzy Deep Convolutional Network.
Old but Still Hot : Several decades ago [Huang et al] near recent
Many real world applications
Convolutional Neural N1etwork: recent developments
Motivation

5. 18 January 2018 IEEE -Tap Energy 2017 5
Introduction to super resolution problem
conventional Approaches
Single Image Super Resolution using Fuzzy Deep Convolutional
Networks
Experimental Results
Conclusion
“Related works”
Conventional SISR Algorithms
1. Manifold learning
2. Dictionary learning
3. Deep learning
• Effectiveness
• Model issues
• Slow Performance
Remaining Challenges

6. 18 January 2018 IEEE -Tap Energy 2017 6
Introduction to super resolution problem
conventional Approaches
Single Image Super Resolution using Fuzzy Deep
Convolutional Networks
Experimental Results
Conclusion
“Proposed Network”
Single Image Super Resolution using Fuzzy Deep Convolutional Networks
Utilize Contextual information spread over very large image region
We employ deep CNN model
To keep pixel-wise info only
Directly learns an end-to-end feature mapping between the low to high-
resolution image.
Key contribution:: incorporation of a fuzzy rule layer with the CNN
structure
Fuzzy rule layer added to CNN
network offers:
• Noise reduction property
• Task-driven feature learning
(preserve spatial coherence).
• Rule-driven Selective patch
processing

7. 18 January 2018 IEEE -Tap Energy 2017 7
Introduction to super resolution problem
conventional Approaches
Single Image Super Resolution using Fuzzy Deep
Convolutional Networks
Experimental Results
Conclusion
“Proposed Network”
Single Image Super Resolution using Fuzzy Deep Convolutional Networks
Our approach for fast convergence and scales
Fuzzy CNN performs hierarchical feature learning between LR image
and final HR image
 Fuzzy deep learning SISR approach comprises of a two-level
reconstruction schema,
(i) Model Construction
(ii) Image Reconstruction
Advantages
• Fast Convergence
• Multi-scale- Train a single
convolutional network to learn and
handle multi-scale factors
Different scale HELPS each other!

8. 18 January 2018 IEEE -Tap Energy 2017 8
Introduction to super resolution problem
conventional Approaches
Single Image Super Resolution using Fuzzy Deep
Convolutional Networks
Experimental Results
Conclusion
“Proposed Network”
Single Image Super Resolution using Fuzzy Deep Convolutional Networks
1. Model Construction
 Cuda-convnet package (caffe package and prototxt) and the caffe
models provide end-to-end machine learning systems
 Generate LR and HR patches from training images
 Sample network model
(fsub−f1−f2−f3+3)2 =(9-1-5) = (9+5−1)2
=169 pixels
 In training , Goal is to minimize
1
𝑛 𝑖=1
𝑛
𝐹 𝑋𝑖; 𝛩 − 𝑌𝑖 2
 Enable High learning rate
SPEC
1. 93 training images
2. Size of sub images=33
3. f1=9, f2=1,f3=5
4. n1=64, n2=32
5. 3 Convolutional layers
6. 3×3 filters, 64 channels

9. 18 January 2018 IEEE -Tap Energy 2017 9
Introduction to super resolution problem
conventional Approaches
Single Image Super Resolution using Fuzzy Deep
Convolutional Networks
Experimental Results
Conclusion
“Proposed Network”
Single Image Super Resolution using Fuzzy Deep Convolutional Networks
2. Image Reconstruction
 Image is reconstructed using the network model to enhance the low-
resolution image.
 The key phases to reconstruct the HR image using fuzzy CNN
Feature extraction and representation
Non-linearity mapping of layers
Accumulation of ultimate layer and fuzzy rule layer to reconstruct
the HR image
Fuzzy rule layer
Feature maps extracted by the
conv1 layer may contain hand-
crafted features or ambiguities, and
hence the fuzzy rule layer is
inserted to stifle anomalies or
contaminations and to enhance
informative features.

10. 18 January 2018 IEEE -Tap Energy 2017 10
Introduction to super resolution problem
conventional Approaches
Single Image Super Resolution using Fuzzy Deep
Convolutional Networks
Experimental Results
Conclusion
“Proposed Network”
Processing Pipeline
Mathematical repesentation
If x is ADJACENT to HR=
𝑔𝑐
𝑥
≈
𝑖𝑓 𝑥 𝑖𝑠 𝐴𝐷𝐽𝐴𝐶𝐸𝑁𝑇 𝑡𝑜 𝑔 𝑐
𝑥
𝐴𝑁𝐷 𝑖𝑓 𝑥2 𝑖𝑠 𝐴𝐷𝐽𝐴𝐶𝐸𝑁𝑇 𝑡𝑜 𝑔 𝑐
𝑥
…..…
𝐴𝑁𝐷 𝑖𝑓 𝑥𝑝 𝑖𝑠 𝐴𝐷𝐽𝐴𝐶𝐸𝑁𝑇 𝑡𝑜 𝑔 𝑛
𝑥

11. 18 January 2018 IEEE -Tap Energy 2017 11
Introduction to super resolution problem
conventional Approaches
Single Image Super Resolution using Fuzzy Deep Convolutional
Networks
Experimental Results
Conclusion
“Experiment Results”
Bicubic
Interpolation
Our
approach
VS.
Sharp
Low
visual
Quality
Even
Sharper
Richer
texture
Visually
pleasing
Detailed
image
Sparse
coding

12. 18 January 2018 IEEE -Tap Energy 2017 12
Introduction to super resolution problem
conventional Approaches
Single Image Super Resolution using Fuzzy Deep Convolutional
Networks
Experimental Results
Conclusion
“Experiment Results”

13. 18 January 2018 IEEE -Tap Energy 2017 13
Introduction to super resolution problem
conventional Approaches
Single Image Super Resolution using Fuzzy Deep Convolutional
Networks
Experimental Results
Conclusion
“Experiment Results”
 Cuda-convnet package used for train a model
 Qualitatively and quantitatively analyzed by;
Peak Signal Noise Ratio (PSNR)
Structural Similarity Index( SSIM)
Feature Similarity Index (FSIM )
 Performance of the auto-learning algorithm compared with bicubic interpolation, Sparse coding, and SRCNN
One model, Multiple-scales

14. 18 January 2018 IEEE -Tap Energy 2017 14
Introduction to super resolution problem
conventional Approaches
Single Image Super Resolution using Fuzzy Deep Convolutional
Networks
Experimental Results
Conclusion
“Experiment Results”
Bicubic interpolation Sparse coding
SRCNN
3
Fuzzy Deep
Learning
Quantitative Analysis

15. 18 January 2018 IEEE -Tap Energy 2017
15
Introduction to super resolution problem
conventional Approaches
Single Image Super Resolution using Fuzzy Deep Convolutional
Networks
Experimental Results
Conclusion
“Experiment Results”
Bicubic interpolation Sparse coding
SRCNN
3
Fuzzy Deep
Learning
Quantitative Analysis
Magnifica
tion
Factor
Algorithms
Measures Bicubic Sparse
Coding
SRCNN Fuzzy
Deep
Learning
3
PSNR(dB)
SSIM
FSIM
27.36
0.744
0.818
27.77
0.770
0.834
28.12
0.792
0.839
28.47
0.810
0.858

16. 18 January 2018 IEEE -Tap Energy 2017
16
Introduction to super resolution problem
conventional Approaches
Single Image Super Resolution using Fuzzy Deep Convolutional
Networks
Experimental Results
Conclusion
“Experiment Results”
Bicubic interpolation Sparse coding
SRCNN
3
Fuzzy Deep
Learning
Quantitative Analysis
Magnifica
tion
Factor
Algorithms
Measures Bicubic Sparse
Coding
SRCNN Fuzzy
Deep
Learning
3
PSNR(dB)
SSIM
FSIM
24.78
0.765
0.854
25.68
0.805
0.885
26.49
0.861
0.928
28.01
0.889
0.943

17. 18 January 2018 IEEE -Tap Energy 2017
17
Introduction to super resolution problem
conventional Approaches
Single Image Super Resolution using Fuzzy Deep Convolutional
Networks
Experimental Results
Conclusion“Findings”
CONCLUSION
•Avoiding additional overhead during learning such as weight sharing, pooling etc.
• Highlighting feature: fuzzy rule layer accumulates with CNN and offers selective patch processing and noise
reduction
• Ability of the proposed method to preserve the structural information in the final HR image with better visual
quality
•Interested in exploring the applicability of proposed method to video super resolution and audio super
resolution

18. 18 January 2018 IEEE -Tap Energy 2017
18
Thank You
International (biennial) Conference on Technological Advancements in Power & Energy– TAP Energy 2017, Amrita University