Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.
IEEE ICME-2019
8:30 – 12:00, July 08
Shanghai, China
Intelligent
Image/Video Editing
Tutorial
STRUCT Group
Jiaying Liu
Wenhan Yang
Retinex Model-Based
Low Light Enhancement
Part 4
Low-Light Image Enhancement
Outline
Background and Related Work / 066
Robust Retinex Model / 085
Sequential Decomposition ...
Low-Light Image Enhancement
Outline
Background and Related Work / 066
Robust Retinex Model / 085
Sequential Decomposition ...
STRUCT GroupBackground
Low-light condition
 Low visibility
 Low contrast
 Intensive noise
5
STRUCT GroupBackground
Simple operations
 e.g. Histogram Equalization
6
STRUCT GroupBackground
Simple operations
 e.g. Histogram Equalization
7
STRUCT Group08
Representative Work
Related Works
Histogram
Equalization
 Enhance the contrast
 Over-enhancement / under-...
STRUCT Group09
Representative Work
Related Works
Histogram
Equalization
Dehazing Method
 Inverted low-light images vs. ha...
STRUCT Group010
Representative Work
Related Works
Histogram
Equalization
Dehazing Method Retinex Model
 Retinex-based met...
STRUCT GroupRelated Works
Retinex model based methods
 Retinex Model
 Estimate L, R = I/L, refine R
11
STRUCT GroupRelated Works
Retinex model based methods
 Retinex Model
 Estimate L, R = I/L, refine R
 NPE[TIP13] Bright-...
STRUCT GroupRelated Works
Retinex model based methods
 Retinex Model
 Estimate L, R = I/L, refine R
 NPE[TIP13]
 Fusio...
STRUCT GroupRelated Works
Retinex model based methods
 Retinex Model
 Estimate L, R = I/L, refine R
 NPE[TIP13]
 Fusio...
STRUCT GroupRelated Works
Retinex model based methods
 Retinex Model
 Estimate L, R = I/L, refine R
 NPE[TIP13]
 Fusio...
STRUCT GroupRelated Works
Retinex model based methods
 Retinex Model
 Estimate L, R = I/L, refine R
 NPE[TIP13]
 Fusio...
STRUCT GroupRelated Works
Retinex model based methods
 Retinex Model
 Estimate L, R = I/L, refine R
 NPE[TIP13]
 Fusio...
STRUCT GroupRelated Works
Retinex model based methods
 Retinex Model
 Estimate L, R = I/L, refine R
 NPE[TIP13]
 Fusio...
STRUCT GroupRelated Works
 Cannot handle noise
Input SRIE[CVPR16]
Retinex model based methods
19
STRUCT Group020
Representative Work
Related Works
Histogram
Equalization
Dehazing Method
Retinex Model
Learning-Based
Meth...
STRUCT GroupRelated Works
Learning based methods
 LLNet[PR17]
 Deep autoencoder
21
STRUCT GroupRelated Works
Learning based methods
 LLCNN[VCIP17]
 Inception module
 Residual learning
 SSIM loss
22
Low-Light Image Enhancement
Outline
Background and Related Work / 066
Robust Retinex Model / 085
Sequential Decomposition ...
STRUCT Group24 Robust Retinex Model for Low Light Enhancement
 Robust Retinex Model for Low Light Enhancement
Structure-R...
STRUCT GroupRobust Retinex Model for Low Light Enhancement
Robust Retinex model
 An additional noise term
 Drawbacks of ...
STRUCT GroupRobust Retinex Model for Low Light Enhancement
Robust Retinex model
 An additional noise term
 Priors for lo...
STRUCT GroupRobust Retinex Model for Low Light Enhancement
Robust Retinex model
 The optimization function
27
STRUCT GroupRobust Retinex Model for Low Light Enhancement
Robust Retinex model
 The optimization function
Data fidelity ...
STRUCT GroupRobust Retinex Model for Low Light Enhancement
Robust Retinex model
 The optimization function
Input image Il...
STRUCT GroupRobust Retinex Model for Low Light Enhancement
Robust Retinex model
 The optimization function
Reflectance co...
STRUCT GroupRobust Retinex Model for Low Light Enhancement
Robust Retinex model
 The optimization function
Without constr...
STRUCT GroupRobust Retinex Model for Low Light Enhancement
Robust Retinex model
 The optimization function
Noise constrai...
STRUCT GroupRobust Retinex Model for Low Light Enhancement
Robust Retinex model
 The solution
 Importing an auxiliary va...
STRUCT GroupRobust Retinex Model for Low Light Enhancement
Robust Retinex model
 The solution
 Sub-problem R
 Sub-probl...
STRUCT GroupRobust Retinex Model for Low Light Enhancement
Robust Retinex model
 The solution
 Sub-problem T
 Updating ...
STRUCT GroupRobust Retinex Model for Low Light Enhancement
Experimental Results
PIE[TIP15] SRIE[CVPR16] Proposed
Input ima...
STRUCT GroupRobust Retinex Model for Low Light Enhancement
Experimental Results
PIE[TIP15] SRIE[CVPR16] Proposed
Input ima...
STRUCT GroupRobust Retinex Model for Low Light Enhancement
Experimental Results
Input image LIME[TIP17]
NPE[TIP13] PIE[TIP...
STRUCT GroupRobust Retinex Model for Low Light Enhancement
Experimental Results
PIE[TIP15]
SRIE[CVPR16] Proposed
Input ima...
STRUCT GroupRobust Retinex Model for Low Light Enhancement
Experimental Results
PIE[TIP15]
SRIE[CVPR16] Proposed
Input ima...
STRUCT GroupRobust Retinex Model for Low Light Enhancement
Experimental Results
Input image LIME[TIP17]
NPE[TIP13] PIE[TIP...
STRUCT GroupRobust Retinex Model for Low Light Enhancement
Experimental Results
 Objective criteria
The lower, the better...
STRUCT GroupRobust Retinex Model for Low Light Enhancement
Experimental Results
Input image
43
STRUCT GroupRobust Retinex Model for Low Light Enhancement
Experimental Results
HE
44
STRUCT GroupRobust Retinex Model for Low Light Enhancement
Experimental Results
CLAHE
45
STRUCT GroupRobust Retinex Model for Low Light Enhancement
Experimental Results
Gamma Correction
46
STRUCT GroupRobust Retinex Model for Low Light Enhancement
Experimental Results
LIME
47
STRUCT GroupRobust Retinex Model for Low Light Enhancement
Experimental Results
NPE
48
STRUCT GroupRobust Retinex Model for Low Light Enhancement
Experimental Results
SRIE
49
STRUCT GroupRobust Retinex Model for Low Light Enhancement
Experimental Results
Proposed method
50
STRUCT GroupRobust Retinex Model for Low Light Enhancement
Experimental Results
ProposedInput image Fu[ICIP14]
 Underwate...
STRUCT GroupRobust Retinex Model for Low Light Enhancement
Experimental Results
 Smoky/hazy image enhancement
ProposedInp...
STRUCT GroupRobust Retinex Model for Low Light Enhancement
Experimental Results
 Enhancement of images taken under dusty ...
Low-Light Image Enhancement
Outline
Background and Related Work / 066
Robust Retinex Model / 085
Sequential Decomposition ...
STRUCT Group55 Sequential Decomposition for Low Light Enhancement
 Sequential Decomposition for Low Light Enhancement
Joi...
STRUCT GroupSequential Decomposition for Low Light Enhancement
Motivation
 Motivation
 Existing methods seldom consider ...
STRUCT GroupSequential Decomposition for Low Light Enhancement
Architecture
In RGB SpaceS
Illumination Estimation
Reflecta...
STRUCT GroupSequential Decomposition for Low Light Enhancement
Illumination Estimation
2
1
ˆarg min || || || ||F
L
L L L...
STRUCT GroupSequential Decomposition for Low Light Enhancement
Reflectance Estimation
 Estimate reflectance based on refi...
STRUCT GroupSequential Decomposition for Low Light Enhancement
Reflectance Estimation
 Use weighted matrices to restrict ...
STRUCT GroupSequential Decomposition for Low Light Enhancement
Solution
 Estimate the illumination map
 Approximate:
 R...
STRUCT GroupSequential Decomposition for Low Light Enhancement
Solution
 Estimate the illumination map
 Estimate the ref...
STRUCT GroupSequential Decomposition for Low Light Enhancement
Experimental Results
 Experimental settings
 All experime...
STRUCT GroupSequential Decomposition for Low Light Enhancement
Experimental Results
 LIME (top panel) and ours (bottom pa...
STRUCT GroupSequential Decomposition for Low Light Enhancement
Experimental Results
 LIME (top panel) and ours (bottom pa...
STRUCT GroupSequential Decomposition for Low Light Enhancement
Experimental Results
 Low-light Image Enhancement Results
...
STRUCT GroupSequential Decomposition for Low Light Enhancement
Experimental Results
 Low-light Image Enhancement Results
...
STRUCT GroupSequential Decomposition for Low Light Enhancement
Experimental Results
 Noisy Low-Light Image Enhancement Re...
STRUCT GroupSequential Decomposition for Low Light Enhancement
Experimental Results
 Noisy Low-Light Image Enhancement Re...
STRUCT GroupSequential Decomposition for Low Light Enhancement
Experimental Results
 Noisy Low-Light Image Enhancement Re...
STRUCT GroupSequential Decomposition for Low Light Enhancement
Experimental Results
 Noisy Low-Light Image Enhancement Re...
STRUCT GroupSequential Decomposition for Low Light Enhancement
Experimental Results
 Noisy Low-Light Image Enhancement Re...
STRUCT GroupSequential Decomposition for Low Light Enhancement
Experimental Results
 Noisy Low-Light Image Enhancement Re...
STRUCT GroupSequential Decomposition for Low Light Enhancement
Summary
 Based on a refined Retinex model
 Noise-removal ...
Low-Light Image Enhancement
Outline
Background and Related Work / 066
Robust Retinex Model / 085
Sequential Decomposition ...
STRUCT Group76 Deep Retinex Decomposition
 Deep Retinex Decomposition
Deep Retinex Decomposition for Low-Light Enhancemen...
STRUCT GroupDeep Retinex Decomposition
Hand-Crafted Retinex
 Hand-crafted constraints and manipulation
 Limited model ca...
STRUCT GroupDeep Retinex Decomposition
Hand-Crafted Retinex
 Not easy to be adaptive to complex and varying low-light
con...
STRUCT GroupDeep Retinex Decomposition
Direct End-to-End Learning
 Difficulties in directly recovering normal-light image...
STRUCT GroupDeep Retinex Decomposition
Direct End-to-End Learning
 Regression to mean
 Over-smoothed results with degrad...
STRUCT GroupDeep Retinex Decomposition
Our Solution: Retinex-Net
 Retinex Theory + Deep Learning
Low-Light Image Dataset
...
STRUCT GroupDeep Retinex Decomposition
Architecture of Retinex-Net
82
STRUCT GroupDeep Retinex Decomposition
Architecture of Retinex-Net
, ,
recon ij i j j
i low normal j low normal
L R I S
...
STRUCT GroupDeep Retinex Decomposition
Architecture of Retinex-Net
, ,
exp( )is j i g j i
i low normal j h v
L I R
 
 ...
STRUCT GroupDeep Retinex Decomposition
Architecture of Retinex-Net
 Training Phase
85
STRUCT GroupDeep Retinex Decomposition
Architecture of Retinex-Net
 Testing Phase
86
STRUCT GroupDeep Retinex Decomposition
Architecture of Retinex-Net
 Training Phase
87
STRUCT GroupDeep Retinex Decomposition
Architecture of Retinex-Net
 Testing Phase
88
STRUCT GroupDeep Retinex Decomposition
Real Photography Pairs
 LOw Light paired dataset (LOL)
 1000 low/normal-light ima...
STRUCT GroupDeep Retinex Decomposition
Dataset
 Synthetic Pairs from Raw Images
 1000 raw images from RAISE[Dang-Nguyen ...
STRUCT GroupDeep Retinex Decomposition
Experiments: Image Decomposition
 Compared Methods
 NPE[Wang2013], Naturalness pr...
STRUCT GroupDeep Retinex Decomposition
Decomposition
Low-Light Image R by LIME I by LIME
Normal-Light Image R by LIME I by...
STRUCT GroupDeep Retinex Decomposition
Decomposition
Low-Light Image R by NPE I by NPE
Normal-Light Image R by NPE I by NP...
STRUCT GroupDeep Retinex Decomposition
Decomposition
Low-Light Image R by Retinex-Net I by Retinex-Net
Normal-Light Image ...
STRUCT GroupDeep Retinex Decomposition
Decomposition
Low-Light Image R by LIME I by LIME
Normal-Light Image R by LIME I by...
STRUCT GroupDeep Retinex Decomposition
Decomposition
Low-Light Image R by NPE I by NPE
Normal-Light Image R by NPE I by NP...
STRUCT GroupDeep Retinex Decomposition
Decomposition
Low-Light Image R by NPE I by NPE
Normal-Light Image R by NPE I by NP...
STRUCT GroupDeep Retinex Decomposition
Experiments: Low-Light Enhancement
 Compared Methods
 DeHz[Dong2011], De-hazing b...
STRUCT GroupDeep Retinex Decomposition
Experiments: Low-Light Enhancement
 Visual Results
99
STRUCT GroupDeep Retinex Decomposition
Experiments: Low-Light Enhancement
 Visual Results
100
STRUCT GroupDeep Retinex Decomposition
Experiments: Low-Light Enhancement
 Visual Results
101
STRUCT GroupDeep Retinex Decomposition
Experiments: Low-Light Enhancement
 Visual Results
102
STRUCT GroupDeep Retinex Decomposition
Experiments: Low-Light Enhancement
 Visual Results
103
STRUCT GroupDeep Retinex Decomposition
Experiments: Low-Light Enhancement
 Visual Results
104
STRUCT GroupDeep Retinex Decomposition
Experiments: Low-Light Enhancement
 Visual Results
105
STRUCT GroupDeep Retinex Decomposition
Experiments: Low-Light Enhancement
 Visual Results
106
STRUCT GroupDeep Retinex Decomposition
Experiments: Low-Light Enhancement
 Visual Results
107
STRUCT GroupDeep Retinex Decomposition
Experiments: Low-Light Enhancement
 Visual Results
108
STRUCT GroupDeep Retinex Decomposition
Experiments: Low-Light Enhancement
 Visual Results
109
STRUCT GroupDeep Retinex Decomposition
Experiments: Low-Light Enhancement
 Visual Results
110
STRUCT Group
liujiaying@pku.edu.cn
yangwenhan@pku.edu.cn
Upcoming SlideShare
Loading in …5
×

Intelligent Image Enhancement and Restoration - From Prior Driven Model to Advanced Deep Learning Part 4: retinex model based low light enhancement

13 views

Published on

ICME2019 Tutorial: Intelligent Image Enhancement and Restoration - From Prior Driven Model to Advanced Deep Learning Part 4: retinex model based low light enhancement

Published in: Internet
  • Be the first to comment

  • Be the first to like this

Intelligent Image Enhancement and Restoration - From Prior Driven Model to Advanced Deep Learning Part 4: retinex model based low light enhancement

  1. 1. IEEE ICME-2019 8:30 – 12:00, July 08 Shanghai, China Intelligent Image/Video Editing Tutorial
  2. 2. STRUCT Group Jiaying Liu Wenhan Yang Retinex Model-Based Low Light Enhancement Part 4
  3. 3. Low-Light Image Enhancement Outline Background and Related Work / 066 Robust Retinex Model / 085 Sequential Decomposition / 115 Deep Retinex Network / 139
  4. 4. Low-Light Image Enhancement Outline Background and Related Work / 066 Robust Retinex Model / 085 Sequential Decomposition / 115 Deep Retinex Network / 139
  5. 5. STRUCT GroupBackground Low-light condition  Low visibility  Low contrast  Intensive noise 5
  6. 6. STRUCT GroupBackground Simple operations  e.g. Histogram Equalization 6
  7. 7. STRUCT GroupBackground Simple operations  e.g. Histogram Equalization 7
  8. 8. STRUCT Group08 Representative Work Related Works Histogram Equalization  Enhance the contrast  Over-enhancement / under-enhancement  Amplify the noise Before HE After HE
  9. 9. STRUCT Group09 Representative Work Related Works Histogram Equalization Dehazing Method  Inverted low-light images vs. hazy images  Invert  dehaze  invert again  Require an additional denoising process Low-Light Inversion Dehazing Result
  10. 10. STRUCT Group010 Representative Work Related Works Histogram Equalization Dehazing Method Retinex Model  Retinex-based methods  Retinex decomposition  Generate results S R L  1 enhanceS R L   Gamma Correction Low-Light Image Enhanced Image Illumination (L) Reflectance (R)
  11. 11. STRUCT GroupRelated Works Retinex model based methods  Retinex Model  Estimate L, R = I/L, refine R 11
  12. 12. STRUCT GroupRelated Works Retinex model based methods  Retinex Model  Estimate L, R = I/L, refine R  NPE[TIP13] Bright-pass filter  preserve naturalness 12
  13. 13. STRUCT GroupRelated Works Retinex model based methods  Retinex Model  Estimate L, R = I/L, refine R  NPE[TIP13]  Fusion based[SP16] Bright-pass filter  preserve naturalness 13
  14. 14. STRUCT GroupRelated Works Retinex model based methods  Retinex Model  Estimate L, R = I/L, refine R  NPE[TIP13]  Fusion based[SP16]  LIME[TIP17] Bright-pass filter  preserve naturalness 14
  15. 15. STRUCT GroupRelated Works Retinex model based methods  Retinex Model  Estimate L, R = I/L, refine R  NPE[TIP13]  Fusion based[SP16]  LIME[TIP17]  Estimate L and R simultaneously Bright-pass filter  preserve naturalness 15
  16. 16. STRUCT GroupRelated Works Retinex model based methods  Retinex Model  Estimate L, R = I/L, refine R  NPE[TIP13]  Fusion based[SP16]  LIME[TIP17]  Estimate L and R simultaneously  PIE[TIP15] Bright-pass filter  preserve naturalness 16
  17. 17. STRUCT GroupRelated Works Retinex model based methods  Retinex Model  Estimate L, R = I/L, refine R  NPE[TIP13]  Fusion based[SP16]  LIME[TIP17]  Estimate L and R simultaneously  PIE[TIP15]  SRIE[CVPR16] Bright-pass filter  preserve naturalness
  18. 18. STRUCT GroupRelated Works Retinex model based methods  Retinex Model  Estimate L, R = I/L, refine R  NPE[TIP13]  Fusion based[SP16]  LIME[TIP17]  Estimate L and R simultaneously  PIE[TIP15]  SRIE[CVPR16]  CEID[TIP17] Bright-pass filter  preserve naturalness 18
  19. 19. STRUCT GroupRelated Works  Cannot handle noise Input SRIE[CVPR16] Retinex model based methods 19
  20. 20. STRUCT Group020 Representative Work Related Works Histogram Equalization Dehazing Method Retinex Model Learning-Based Method Low-Light Image Dataset Regression Model OutputInput Low-Light Image Dataset …
  21. 21. STRUCT GroupRelated Works Learning based methods  LLNet[PR17]  Deep autoencoder 21
  22. 22. STRUCT GroupRelated Works Learning based methods  LLCNN[VCIP17]  Inception module  Residual learning  SSIM loss 22
  23. 23. Low-Light Image Enhancement Outline Background and Related Work / 066 Robust Retinex Model / 085 Sequential Decomposition / 115 Deep Retinex Network / 139
  24. 24. STRUCT Group24 Robust Retinex Model for Low Light Enhancement  Robust Retinex Model for Low Light Enhancement Structure-Revealing Low-Light Image Enhancement Via Robust Retinex Model Mading Li, Jiaying Liu, Wenhan Yang, Xiaoyan Sun, and Zongming Guo TIP 2018 I R L  I R L N Input image Retinex Model Robust Retinex Model
  25. 25. STRUCT GroupRobust Retinex Model for Low Light Enhancement Robust Retinex model  An additional noise term  Drawbacks of conventional model  Focus on the estimation of L  Got noisy reflectance for  Calculate both R and L iteratively  Introduce noise to illumination by minimizing 25 ' /R R N L  2 || ||FR L S
  26. 26. STRUCT GroupRobust Retinex Model for Low Light Enhancement Robust Retinex model  An additional noise term  Priors for low-light images  Illumination map  piece-wise smoothed  Reflectance map  low contrast  Noise map  relatively low intensity 26
  27. 27. STRUCT GroupRobust Retinex Model for Low Light Enhancement Robust Retinex model  The optimization function 27
  28. 28. STRUCT GroupRobust Retinex Model for Low Light Enhancement Robust Retinex model  The optimization function Data fidelity term 28
  29. 29. STRUCT GroupRobust Retinex Model for Low Light Enhancement Robust Retinex model  The optimization function Input image Illumination map Illumination constraint 29
  30. 30. STRUCT GroupRobust Retinex Model for Low Light Enhancement Robust Retinex model  The optimization function Reflectance constraint 30
  31. 31. STRUCT GroupRobust Retinex Model for Low Light Enhancement Robust Retinex model  The optimization function Without constraint With constraint Reflectance constraint 31
  32. 32. STRUCT GroupRobust Retinex Model for Low Light Enhancement Robust Retinex model  The optimization function Noise constraint Input image w/o constraint w/ constraint Noise map 32
  33. 33. STRUCT GroupRobust Retinex Model for Low Light Enhancement Robust Retinex model  The solution  Importing an auxiliary variable T  Augmented Lagrange equation 33
  34. 34. STRUCT GroupRobust Retinex Model for Low Light Enhancement Robust Retinex model  The solution  Sub-problem R  Sub-problem L  Sub-problem N 34
  35. 35. STRUCT GroupRobust Retinex Model for Low Light Enhancement Robust Retinex model  The solution  Sub-problem T  Updating auxiliary variables 35
  36. 36. STRUCT GroupRobust Retinex Model for Low Light Enhancement Experimental Results PIE[TIP15] SRIE[CVPR16] Proposed Input image LIME[TIP17] NPE[TIP13] 36
  37. 37. STRUCT GroupRobust Retinex Model for Low Light Enhancement Experimental Results PIE[TIP15] SRIE[CVPR16] Proposed Input image LIME[TIP17] NPE[TIP13] 37
  38. 38. STRUCT GroupRobust Retinex Model for Low Light Enhancement Experimental Results Input image LIME[TIP17] NPE[TIP13] PIE[TIP15] SRIE[CVPR16] ProposedHE 38
  39. 39. STRUCT GroupRobust Retinex Model for Low Light Enhancement Experimental Results PIE[TIP15] SRIE[CVPR16] Proposed Input image LIME[TIP17] NPE[TIP13] 39
  40. 40. STRUCT GroupRobust Retinex Model for Low Light Enhancement Experimental Results PIE[TIP15] SRIE[CVPR16] Proposed Input image LIME[TIP17] NPE[TIP13] 40
  41. 41. STRUCT GroupRobust Retinex Model for Low Light Enhancement Experimental Results Input image LIME[TIP17] NPE[TIP13] PIE[TIP15] SRIE[CVPR16] ProposedHE 41
  42. 42. STRUCT GroupRobust Retinex Model for Low Light Enhancement Experimental Results  Objective criteria The lower, the better quality The higher, the better quality 42
  43. 43. STRUCT GroupRobust Retinex Model for Low Light Enhancement Experimental Results Input image 43
  44. 44. STRUCT GroupRobust Retinex Model for Low Light Enhancement Experimental Results HE 44
  45. 45. STRUCT GroupRobust Retinex Model for Low Light Enhancement Experimental Results CLAHE 45
  46. 46. STRUCT GroupRobust Retinex Model for Low Light Enhancement Experimental Results Gamma Correction 46
  47. 47. STRUCT GroupRobust Retinex Model for Low Light Enhancement Experimental Results LIME 47
  48. 48. STRUCT GroupRobust Retinex Model for Low Light Enhancement Experimental Results NPE 48
  49. 49. STRUCT GroupRobust Retinex Model for Low Light Enhancement Experimental Results SRIE 49
  50. 50. STRUCT GroupRobust Retinex Model for Low Light Enhancement Experimental Results Proposed method 50
  51. 51. STRUCT GroupRobust Retinex Model for Low Light Enhancement Experimental Results ProposedInput image Fu[ICIP14]  Underwater image enhancement 51
  52. 52. STRUCT GroupRobust Retinex Model for Low Light Enhancement Experimental Results  Smoky/hazy image enhancement ProposedInput image He[CVPR09] 52
  53. 53. STRUCT GroupRobust Retinex Model for Low Light Enhancement Experimental Results  Enhancement of images taken under dusty weather ProposedInput image Fu[MMSP14] 53
  54. 54. Low-Light Image Enhancement Outline Background and Related Work / 066 Robust Retinex Model / 085 Sequential Decomposition / 115 Deep Retinex Network / 139
  55. 55. STRUCT Group55 Sequential Decomposition for Low Light Enhancement  Sequential Decomposition for Low Light Enhancement Joint Enhancement and Denoising Method via Sequential Decomposition Xutong Ren, Mading Li, Wen-Huang Cheng, and Jiaying Liu ISCAS 2018
  56. 56. STRUCT GroupSequential Decomposition for Low Light Enhancement Motivation  Motivation  Existing methods seldom consider noise  Enhancement procedure  Amplifies existing noise Low-Light Image NPEA 56
  57. 57. STRUCT GroupSequential Decomposition for Low Light Enhancement Architecture In RGB SpaceS Illumination Estimation Reflectance Estimation L R S’ ෠𝐿 W G Restrict Matrices L’ 57
  58. 58. STRUCT GroupSequential Decomposition for Low Light Enhancement Illumination Estimation 2 1 ˆarg min || || || ||F L L L L   Low-Light Image Initial Illumination Estimated Illumination  Estimate illumination independent from reflectance 58
  59. 59. STRUCT GroupSequential Decomposition for Low Light Enhancement Reflectance Estimation  Estimate reflectance based on refined illumination and original image 2 2 2 arg min || / || || || || ||F F F R R S L W R R G       Low-Light Image S / L Estimated Reflectance 59
  60. 60. STRUCT GroupSequential Decomposition for Low Light Enhancement Reflectance Estimation  Use weighted matrices to restrict noise ˆ| |/ ˆ(1 ) 0, if | |ˆ , otherwise S G e S S S S                  1 | | W S eps    G W 60
  61. 61. STRUCT GroupSequential Decomposition for Low Light Enhancement Solution  Estimate the illumination map  Approximate:  Rewrite the original problem:  Simplify: 2 1 x {h,v} ( ( )) || || . ˆ| L( ) | d d d L x L x eps        2 2 x {h,v} ( ( ))ˆarg min || || . ˆ| L( ) | d F L d d L x L L x eps          2 2 x {h,v} ˆarg min || || ( ) ( ( )) .F d d L d L L A x L x       61
  62. 62. STRUCT GroupSequential Decomposition for Low Light Enhancement Solution  Estimate the illumination map  Estimate the reflectance map {h,v} ˆDiag( )T d d d d I D a D l l          {h,v} {h,v} {h,v} Diag( ) / T T d d d d d d d T d d d I D w D D D r s l D g                   62
  63. 63. STRUCT GroupSequential Decomposition for Low Light Enhancement Experimental Results  Experimental settings  All experiments are performed in MATLAB R2017a with 4G RAM and Intel Core i5-4210H CPU @2.90GHz.  In our experiment the parameters α, β and γ are empirically set as 0.007, 0.001 and 0.016.  In our experiment the parameters ε and σ are set to 10 and λ is set to 6. 63
  64. 64. STRUCT GroupSequential Decomposition for Low Light Enhancement Experimental Results  LIME (top panel) and ours (bottom panel) Input images Illumination Reflectance Result images Details 64
  65. 65. STRUCT GroupSequential Decomposition for Low Light Enhancement Experimental Results  LIME (top panel) and ours (bottom panel) Input images Illumination Reflectance Result images Details 65
  66. 66. STRUCT GroupSequential Decomposition for Low Light Enhancement Experimental Results  Low-light Image Enhancement Results (a) Input (b) HE (c) SRIE (d) NPEA (e) LIME (f ) Our Method 66
  67. 67. STRUCT GroupSequential Decomposition for Low Light Enhancement Experimental Results  Low-light Image Enhancement Results (a) Input (b) HE (c) SRIE (d) NPEA (e) LIME (f ) Our Method 67
  68. 68. STRUCT GroupSequential Decomposition for Low Light Enhancement Experimental Results  Noisy Low-Light Image Enhancement Results Input 68
  69. 69. STRUCT GroupSequential Decomposition for Low Light Enhancement Experimental Results  Noisy Low-Light Image Enhancement Results PIE 69
  70. 70. STRUCT GroupSequential Decomposition for Low Light Enhancement Experimental Results  Noisy Low-Light Image Enhancement Results SRIE 70
  71. 71. STRUCT GroupSequential Decomposition for Low Light Enhancement Experimental Results  Noisy Low-Light Image Enhancement Results LIME 71
  72. 72. STRUCT GroupSequential Decomposition for Low Light Enhancement Experimental Results  Noisy Low-Light Image Enhancement Results NPEA 72
  73. 73. STRUCT GroupSequential Decomposition for Low Light Enhancement Experimental Results  Noisy Low-Light Image Enhancement Results Our Method 73
  74. 74. STRUCT GroupSequential Decomposition for Low Light Enhancement Summary  Based on a refined Retinex model  Noise-removal and Enhancing  Sequential decomposition 74
  75. 75. Low-Light Image Enhancement Outline Background and Related Work / 066 Robust Retinex Model / 085 Sequential Decomposition / 115 Deep Retinex Network / 139
  76. 76. STRUCT Group76 Deep Retinex Decomposition  Deep Retinex Decomposition Deep Retinex Decomposition for Low-Light Enhancement Chen Wei*, Wenjing Wang*, Wenhan Yang, Jiaying Liu * indicates equal contributions BMVC 2018
  77. 77. STRUCT GroupDeep Retinex Decomposition Hand-Crafted Retinex  Hand-crafted constraints and manipulation  Limited model capacity OutputInput Retinex Decomposition Adjusted Decomposition 77
  78. 78. STRUCT GroupDeep Retinex Decomposition Hand-Crafted Retinex  Not easy to be adaptive to complex and varying low-light conditions Under-enhancementOver-enhancement Boundary artifacts 78
  79. 79. STRUCT GroupDeep Retinex Decomposition Direct End-to-End Learning  Difficulties in directly recovering normal-light images Inherent ambiguity Low-Light Image Dataset Regression Model OutputInput Low-Light Image Dataset … 79
  80. 80. STRUCT GroupDeep Retinex Decomposition Direct End-to-End Learning  Regression to mean  Over-smoothed results with degraded contrast Over-smoothness Degraded contrast 80
  81. 81. STRUCT GroupDeep Retinex Decomposition Our Solution: Retinex-Net  Retinex Theory + Deep Learning Low-Light Image Dataset OutputInput Retinex Decomposition Adjusted Decomposition Regression Model … 81
  82. 82. STRUCT GroupDeep Retinex Decomposition Architecture of Retinex-Net 82
  83. 83. STRUCT GroupDeep Retinex Decomposition Architecture of Retinex-Net , , recon ij i j j i low normal j low normal L R I S      ir low normalL R R  , , exp( )is j i g j i i low normal j h v L I R         Reconstruction Loss  Constant Reflectance Loss  Illumination Smoothness Loss 83
  84. 84. STRUCT GroupDeep Retinex Decomposition Architecture of Retinex-Net , , exp( )is j i g j i i low normal j h v L I R        , is i i low normal L I     Illumination Smoothness Loss 84
  85. 85. STRUCT GroupDeep Retinex Decomposition Architecture of Retinex-Net  Training Phase 85
  86. 86. STRUCT GroupDeep Retinex Decomposition Architecture of Retinex-Net  Testing Phase 86
  87. 87. STRUCT GroupDeep Retinex Decomposition Architecture of Retinex-Net  Training Phase 87
  88. 88. STRUCT GroupDeep Retinex Decomposition Architecture of Retinex-Net  Testing Phase 88
  89. 89. STRUCT GroupDeep Retinex Decomposition Real Photography Pairs  LOw Light paired dataset (LOL)  1000 low/normal-light image pairs  500 are collected by changing only exposure time and ISO  Various scenes, e.g., houses, clubs, streets, etc. 89
  90. 90. STRUCT GroupDeep Retinex Decomposition Dataset  Synthetic Pairs from Raw Images  1000 raw images from RAISE[Dang-Nguyen 2015]  Fitting the histogram of Y channel in YCbCr to real low-light images  Online available: https://daooshee.github.io/BMVC2018website/ 90
  91. 91. STRUCT GroupDeep Retinex Decomposition Experiments: Image Decomposition  Compared Methods  NPE[Wang2013], Naturalness preserved enhancement algorithm  LIME[Guo2017], Illumination Estimation based method  Evaluation Dataset  LOL, Evaluation set of LOL dataset, containing 50 images 91
  92. 92. STRUCT GroupDeep Retinex Decomposition Decomposition Low-Light Image R by LIME I by LIME Normal-Light Image R by LIME I by LIME 92
  93. 93. STRUCT GroupDeep Retinex Decomposition Decomposition Low-Light Image R by NPE I by NPE Normal-Light Image R by NPE I by NPE 93
  94. 94. STRUCT GroupDeep Retinex Decomposition Decomposition Low-Light Image R by Retinex-Net I by Retinex-Net Normal-Light Image R by Retinex-Net I by Retinex-Net 94
  95. 95. STRUCT GroupDeep Retinex Decomposition Decomposition Low-Light Image R by LIME I by LIME Normal-Light Image R by LIME I by LIME 95
  96. 96. STRUCT GroupDeep Retinex Decomposition Decomposition Low-Light Image R by NPE I by NPE Normal-Light Image R by NPE I by NPE 96
  97. 97. STRUCT GroupDeep Retinex Decomposition Decomposition Low-Light Image R by NPE I by NPE Normal-Light Image R by NPE I by NPE 97
  98. 98. STRUCT GroupDeep Retinex Decomposition Experiments: Low-Light Enhancement  Compared Methods  DeHz[Dong2011], De-hazing based method  NPE[Dong2011], Naturalness preserved enhancement algorithm  SRIE[Fu2016], Simultaneous Reflection and Illumination Estimation  LIME[Guo2017], Illumination Estimation based method  Evaluation Dataset  LIME[Guo2017], 10 low-light images  MEF[Guo2017], 17 images sequences with multiple exposure levels  DICM[Lee2013], 69 captured images with commercial digital cameras 98
  99. 99. STRUCT GroupDeep Retinex Decomposition Experiments: Low-Light Enhancement  Visual Results 99
  100. 100. STRUCT GroupDeep Retinex Decomposition Experiments: Low-Light Enhancement  Visual Results 100
  101. 101. STRUCT GroupDeep Retinex Decomposition Experiments: Low-Light Enhancement  Visual Results 101
  102. 102. STRUCT GroupDeep Retinex Decomposition Experiments: Low-Light Enhancement  Visual Results 102
  103. 103. STRUCT GroupDeep Retinex Decomposition Experiments: Low-Light Enhancement  Visual Results 103
  104. 104. STRUCT GroupDeep Retinex Decomposition Experiments: Low-Light Enhancement  Visual Results 104
  105. 105. STRUCT GroupDeep Retinex Decomposition Experiments: Low-Light Enhancement  Visual Results 105
  106. 106. STRUCT GroupDeep Retinex Decomposition Experiments: Low-Light Enhancement  Visual Results 106
  107. 107. STRUCT GroupDeep Retinex Decomposition Experiments: Low-Light Enhancement  Visual Results 107
  108. 108. STRUCT GroupDeep Retinex Decomposition Experiments: Low-Light Enhancement  Visual Results 108
  109. 109. STRUCT GroupDeep Retinex Decomposition Experiments: Low-Light Enhancement  Visual Results 109
  110. 110. STRUCT GroupDeep Retinex Decomposition Experiments: Low-Light Enhancement  Visual Results 110
  111. 111. STRUCT Group liujiaying@pku.edu.cn yangwenhan@pku.edu.cn

×