SlideShare a Scribd company logo
1 of 25
Download to read offline
Colorization with total variation regularization
J´ulio Peixoto da Silva J´unior
Fazal-E-Asim
Weskley Vinicius Fernandes Mauricio
Yosbel Rodrigues Ortega
Departamento de Teleinform´atica - UFC
Non-Linear Optimization
July 19, 2017
DETI-UFC Non-Linear Optimization July 19, 2017 1 / 25
Outlines
1 Problem Statement
2 Total Variation Regularization
3 Examples using Boyd’s method (CVX)
4 TV denoising solution.
5 Examples using TV denoising Solution (CVX)
6 Colorization by propagation method
7 Examples using propagation method
8 Comparison and Analysis
9 Conclusions
DETI-UFC Non-Linear Optimization July 19, 2017 2 / 25
Problem Statement
A color image (mxn) is represented as three matrices of intensities
R,G,B ∈ Rmxn with entries in [0,1] representing the red, green, and
blue pixel intensities, respectively.
A color image is converted to a monochrome image, represented as
one matrix M ∈ Rmxn using
M = 0.299R + 0.587G + 0.114B (1)
DETI-UFC Non-Linear Optimization July 19, 2017 3 / 25
Cont’d
In colorization, we are given M, the monochrome version of an image,
and the color values of some of the pixels. We are to guess its color
version,i.e.,the matrices R,G,B.
We have to minimize the total variation function, defined as:
tv(R, G, B) =
m−1
i=1
n−1
j=1








Rij − Ri,j+1
Gij − Gi,j+1
Bij − Bi,j+1
Rij − Ri+1,j
Gij − Gi+1,j
Bij − Bi+1,j








2
(2)
DETI-UFC Non-Linear Optimization July 19, 2017 4 / 25
Cont’d
Subject to consistency with the given monochrome image M, the
knows ranges of the entries of (R,G,B) (i.e.,∈ [0,1]), and the given
color entries.
Monochrome version of the image, M, along with vectors of known
color intensities is given to us.
The tv function, invoked as tv(R,G,B), gives the total variation.
Report your optimal objective value and if you have access to a color
printer, attach your reconstructed image.
If you don’t have access to a color printer, it’s OK to just give the
optimal objective value.
DETI-UFC Non-Linear Optimization July 19, 2017 5 / 25
Cont’d
Convex Problem
Every norms on Rn is convex!
minimize tv(R, G, B)
subject to:
R(knom ind) == R know
G(knom ind) == G know
B(knom ind) == B know
0.229R + 0.587G + 0.114B == M
R, G and B ∈ [0, 1] (3)
DETI-UFC Non-Linear Optimization July 19, 2017 6 / 25
Examples using Boyd’s method (CVX)
Read the image
Read the image flower.png (50x50)
Grayscale Image
Apply M = 0.229R + 0.587G + 0.114 ∗ B to obtain grayscale image
Create Damage Image
known ind = find(rand(m, n) >= 0.90)
Solver
Using CVX for solve the problem
Reconstructed Image
Using the results of CVX for reconstructed the image.
DETI-UFC Non-Linear Optimization July 19, 2017 7 / 25
Results with CVX - Exampe I
(a) Original (b) Monochrome
(c) Given (d) Reconstructed
DETI-UFC Non-Linear Optimization July 19, 2017 8 / 25
Results with CVX - Exampe II
(e) Original (f) Monochrome
(g) Given (h) Reconstructed
DETI-UFC Non-Linear Optimization July 19, 2017 9 / 25
TV denoising solution
minimize ||X − Y || + λ · tv(R, G, B)
subject to:
R(knom ind) == R know
G(knom ind) == G know
B(knom ind) == B know
0.229R + 0.587G + 0.114B == M
R, G and B ∈ [0, 1] (4)
Definition of variables
where X is the concatenation of R, G and B and Y is the monochromatic
image, in vector form.
λ is a positive scale factor.
DETI-UFC Non-Linear Optimization July 19, 2017 10 / 25
Examples using TV denoising solution - Lambda = 0.1
(i) Original (j) Monochrome
(k) Given (l) Reconstructed
DETI-UFC Non-Linear Optimization July 19, 2017 11 / 25
Examples using TV denoising solution - Lambda = 1
(m) Original (n) Monochrome
(o) Given (p) Reconstructed
DETI-UFC Non-Linear Optimization July 19, 2017 12 / 25
Examples using TV denoising solution - Lambda = 10
(q) Original (r) Monochrome
(s) Given (t) Reconstructed
DETI-UFC Non-Linear Optimization July 19, 2017 13 / 25
Colorization by Propagation
YUV color space, is a model commonly used in video, where Y is the
monochromatic luminance channel (intensity), while U and V are the
chrominance channels, encoding the color.
The method called Propagation using Optimization, is given as input
an intensity volume Y (x; y; t) and outputs two color volumes U (x; y;
t) and V (x; y; t).
Figure 1: YUV color space
Thus, if Y(r) is the intensity of a particular pixel, then we wish to
impose the constraint that two neighboring pixels r, s should have
similar colors if their intensities are similar.
DETI-UFC Non-Linear Optimization July 19, 2017 14 / 25
Cont’d
Neighboring pixels with similar intensities should have similar colors,
so, the colors can be propagated to all pixels from known color pixel.
Figure 2: Neighboring pixels.
DETI-UFC Non-Linear Optimization July 19, 2017 15 / 25
Cont’d
The problem results in minimize the difference between the color U(r)
at pixel r and the affinity-weighted average of the colors at
neighboring pixels (s):
Objective Function
minimize J(U) =
r
U(r) −
s∈N(r)
wrsU(s) (5)
where wrs is a affinity-weighting function that sums to one, large
when Y(r) is similar to Y (s), and small when the two intensities are
different. The notation s ∈ N(r) denotes the fact that r and s are
neighboring pixels.
DETI-UFC Non-Linear Optimization July 19, 2017 16 / 25
Cont’d
There are in the literature different types of affinity-weighting
functions. The simplest one is commonly used by image segmentation
algorithms and is based on the squared difference between the two
intensities:
wrs ∝ exp −
(Y (r) − Y (s))2
2σ2
r
(6)
σ proportional to local variance.
Figure 3: Affinity-weighting functions.
DETI-UFC Non-Linear Optimization July 19, 2017 17 / 25
Results with Colorization by Propagation - Example I
(a) Original (b) Monochrome
(c) Given (d) Reconstructed
DETI-UFC Non-Linear Optimization July 19, 2017 18 / 25
Results with Colorization by Propagation - Exampe II
(e) Original (f) Monochrome
(g) Given (h) Reconstructed
DETI-UFC Non-Linear Optimization July 19, 2017 19 / 25
Metrics used for Comparison
Mean Square Error (MSE)
MSE =
1
MN
M
y=1
N
x=1
[I(x, y) − ˜I(x, y)]2
(7)
Peak Signal to Noise Ratio (PSNR)
PSNR = 20 ∗ log10
255
√
MSE
(8)
DETI-UFC Non-Linear Optimization July 19, 2017 20 / 25
Comparison of different methods
TV function Value of Original Image
411.9606
Table 1: Results example I
- Boyd Julia Propagation Denoising
Solver SDPT31, SeDuMi2 SCS3 - SDPT3
TV function 341.47 341.73 383.43 342.24
Time(s) 2.62 7.58 0.3067 4.20
MSE 0.0058 0.0057 0.0022 0.0056
PSNR(dB) 70.49 70.57 74.70 70.64
1
A MATLAB software for semidefinite-quadratic-linear programming.
2
Optimization over symmetric cones.
3
Solver for semidefinite, second-order and exponential cone programming.
DETI-UFC Non-Linear Optimization July 19, 2017 21 / 25
Cont’d ...
TV Function Value of Original Image
28560
Table 2: Results example II
- Matlab Julia Propagation Denoising∗
Solver SDPT3, SeDuMi SCS4 - SDTP3
TV function 25539 25150 28126 -
Time(s) 746,3 3547.13 38,0 -
MSE 0.0015 0.0018 0.000159 -
PSNR(dB) 76.36 75.5781 86.1168 -
4
Solution with intermittent behavior, only one time solved with success.
DETI-UFC Non-Linear Optimization July 19, 2017 22 / 25
Conclusions
We solved the problem using Boyd’s method using MATLAB and
JULIA.
We also used TV denoising and colorization by propagation methods
to solve the same problem in MATLAB.
The propagation method have the best solution as compared to other
methods.
The Boyd’s method shows least performance as compared to other
method..
DETI-UFC Non-Linear Optimization July 19, 2017 23 / 25
References
1 S. Boyd and L. Vandenberghe. Convex Optimization. Cambridge
University Press, 2004.
2 S. Boyd, Jenny Hong el al. Convex Optimization in Julia. HPTCDL
November 16-21, 2014, New Orleans, Louisiana, USA
3 Karanveer Mohan, Madeleine Udell, David Zeng, Jenny Hong.
Convex.jl Documentation. Jun 24, 2017.
4 Nonlinear total variation based noise removal algorithms, Rudin, L. I.;
Osher, S.; Fatemi, E. (1992). Physica D 60: 259–268.
5 A generalized vector-valued total variation algorithm., Rodriguez
Paul, and Brendt Wohlberg. Image Processing (ICIP), 2009 16th
IEEE International Conference on. IEEE, 2009.
6 R´emi Flamary. Avaliable at:
http://remi.flamary.com/demos/proxtv.html [Acessed - 07/16]
7 LEVIN, A., LISCHINSKI, D., AND WEISS, Y. 2004. Colorization
using optimization. ACM Trans. Graph. 23, 3 (Aug.), 689–694.
DETI-UFC Non-Linear Optimization July 19, 2017 24 / 25
The End
Thank you. Gracias. Obrigado
DETI-UFC Non-Linear Optimization July 19, 2017 25 / 25

More Related Content

What's hot

Histogram equalization
Histogram equalizationHistogram equalization
Histogram equalization11mr11mahesh
 
LAPLACE TRANSFORM SUITABILITY FOR IMAGE PROCESSING
LAPLACE TRANSFORM SUITABILITY FOR IMAGE PROCESSINGLAPLACE TRANSFORM SUITABILITY FOR IMAGE PROCESSING
LAPLACE TRANSFORM SUITABILITY FOR IMAGE PROCESSINGPriyanka Rathore
 
Multimedia color in image and video
Multimedia color in image and videoMultimedia color in image and video
Multimedia color in image and videoMazin Alwaaly
 
Lec-08 Feature Aggregation II: Fisher Vector, AKULA and Super Vector
Lec-08 Feature Aggregation II: Fisher Vector, AKULA and Super VectorLec-08 Feature Aggregation II: Fisher Vector, AKULA and Super Vector
Lec-08 Feature Aggregation II: Fisher Vector, AKULA and Super VectorUnited States Air Force Academy
 
Lect 03 - first portion
Lect 03 - first portionLect 03 - first portion
Lect 03 - first portionMoe Moe Myint
 
Iaetsd vlsi implementation of gabor filter based image edge detection
Iaetsd vlsi implementation of gabor filter based image edge detectionIaetsd vlsi implementation of gabor filter based image edge detection
Iaetsd vlsi implementation of gabor filter based image edge detectionIaetsd Iaetsd
 
Color image processing Presentation
Color image processing PresentationColor image processing Presentation
Color image processing PresentationRevanth Chimmani
 
Color to Gray and back’ using normalization of color components with Cosine, ...
Color to Gray and back’ using normalization of color components with Cosine, ...Color to Gray and back’ using normalization of color components with Cosine, ...
Color to Gray and back’ using normalization of color components with Cosine, ...IOSR Journals
 
Image sampling and quantization
Image sampling and quantizationImage sampling and quantization
Image sampling and quantizationBCET, Balasore
 
Comparison of Histogram Equalization Techniques for Image Enhancement of Gray...
Comparison of Histogram Equalization Techniques for Image Enhancement of Gray...Comparison of Histogram Equalization Techniques for Image Enhancement of Gray...
Comparison of Histogram Equalization Techniques for Image Enhancement of Gray...IJMER
 
Histogram Operation in Image Processing
Histogram Operation in Image ProcessingHistogram Operation in Image Processing
Histogram Operation in Image ProcessingVARUN KUMAR
 
Welcome to International Journal of Engineering Research and Development (IJERD)
Welcome to International Journal of Engineering Research and Development (IJERD)Welcome to International Journal of Engineering Research and Development (IJERD)
Welcome to International Journal of Engineering Research and Development (IJERD)IJERD Editor
 
Color models in Digitel image processing
Color models in Digitel image processingColor models in Digitel image processing
Color models in Digitel image processingAryan Shivhare
 
Digital image processing using matlab: basic transformations, filters and ope...
Digital image processing using matlab: basic transformations, filters and ope...Digital image processing using matlab: basic transformations, filters and ope...
Digital image processing using matlab: basic transformations, filters and ope...thanh nguyen
 
Image enhancement using alpha rooting based hybrid technique
Image enhancement using alpha rooting based hybrid techniqueImage enhancement using alpha rooting based hybrid technique
Image enhancement using alpha rooting based hybrid techniqueRahul Yadav
 

What's hot (19)

upgrade2013
upgrade2013upgrade2013
upgrade2013
 
Histogram equalization
Histogram equalizationHistogram equalization
Histogram equalization
 
LAPLACE TRANSFORM SUITABILITY FOR IMAGE PROCESSING
LAPLACE TRANSFORM SUITABILITY FOR IMAGE PROCESSINGLAPLACE TRANSFORM SUITABILITY FOR IMAGE PROCESSING
LAPLACE TRANSFORM SUITABILITY FOR IMAGE PROCESSING
 
Multimedia color in image and video
Multimedia color in image and videoMultimedia color in image and video
Multimedia color in image and video
 
Lec-08 Feature Aggregation II: Fisher Vector, AKULA and Super Vector
Lec-08 Feature Aggregation II: Fisher Vector, AKULA and Super VectorLec-08 Feature Aggregation II: Fisher Vector, AKULA and Super Vector
Lec-08 Feature Aggregation II: Fisher Vector, AKULA and Super Vector
 
Lect 03 - first portion
Lect 03 - first portionLect 03 - first portion
Lect 03 - first portion
 
Iaetsd vlsi implementation of gabor filter based image edge detection
Iaetsd vlsi implementation of gabor filter based image edge detectionIaetsd vlsi implementation of gabor filter based image edge detection
Iaetsd vlsi implementation of gabor filter based image edge detection
 
Color image processing Presentation
Color image processing PresentationColor image processing Presentation
Color image processing Presentation
 
Color to Gray and back’ using normalization of color components with Cosine, ...
Color to Gray and back’ using normalization of color components with Cosine, ...Color to Gray and back’ using normalization of color components with Cosine, ...
Color to Gray and back’ using normalization of color components with Cosine, ...
 
Histogram Equalization
Histogram EqualizationHistogram Equalization
Histogram Equalization
 
Image sampling and quantization
Image sampling and quantizationImage sampling and quantization
Image sampling and quantization
 
Comparison of Histogram Equalization Techniques for Image Enhancement of Gray...
Comparison of Histogram Equalization Techniques for Image Enhancement of Gray...Comparison of Histogram Equalization Techniques for Image Enhancement of Gray...
Comparison of Histogram Equalization Techniques for Image Enhancement of Gray...
 
Histogram Operation in Image Processing
Histogram Operation in Image ProcessingHistogram Operation in Image Processing
Histogram Operation in Image Processing
 
Welcome to International Journal of Engineering Research and Development (IJERD)
Welcome to International Journal of Engineering Research and Development (IJERD)Welcome to International Journal of Engineering Research and Development (IJERD)
Welcome to International Journal of Engineering Research and Development (IJERD)
 
Deep Learning Opening Workshop - Horseshoe Regularization for Machine Learnin...
Deep Learning Opening Workshop - Horseshoe Regularization for Machine Learnin...Deep Learning Opening Workshop - Horseshoe Regularization for Machine Learnin...
Deep Learning Opening Workshop - Horseshoe Regularization for Machine Learnin...
 
Color models in Digitel image processing
Color models in Digitel image processingColor models in Digitel image processing
Color models in Digitel image processing
 
Digital image processing using matlab: basic transformations, filters and ope...
Digital image processing using matlab: basic transformations, filters and ope...Digital image processing using matlab: basic transformations, filters and ope...
Digital image processing using matlab: basic transformations, filters and ope...
 
Image restoration and reconstruction
Image restoration and reconstructionImage restoration and reconstruction
Image restoration and reconstruction
 
Image enhancement using alpha rooting based hybrid technique
Image enhancement using alpha rooting based hybrid techniqueImage enhancement using alpha rooting based hybrid technique
Image enhancement using alpha rooting based hybrid technique
 

Similar to Colorization with total variantion regularization

Iisrt zzz bhavyasri vanteddu
Iisrt zzz bhavyasri vantedduIisrt zzz bhavyasri vanteddu
Iisrt zzz bhavyasri vantedduIISRT
 
Lect 02 second portion
Lect 02  second portionLect 02  second portion
Lect 02 second portionMoe Moe Myint
 
A Four Directions Variational Method For Solving Image Processing Problems
A Four Directions Variational Method For Solving Image Processing ProblemsA Four Directions Variational Method For Solving Image Processing Problems
A Four Directions Variational Method For Solving Image Processing ProblemsClaudia Acosta
 
Alternating direction-method-for-image-restoration
Alternating direction-method-for-image-restorationAlternating direction-method-for-image-restoration
Alternating direction-method-for-image-restorationPrashant Pal
 
Lec_2_Digital Image Fundamentals.pdf
Lec_2_Digital Image Fundamentals.pdfLec_2_Digital Image Fundamentals.pdf
Lec_2_Digital Image Fundamentals.pdfnagwaAboElenein
 
Fundamentals of image processing
Fundamentals of image processing  Fundamentals of image processing
Fundamentals of image processing BCET, Balasore
 
On image intensities, eigenfaces and LDA
On image intensities, eigenfaces and LDAOn image intensities, eigenfaces and LDA
On image intensities, eigenfaces and LDARaghu Palakodety
 
Contrast enhancement using various statistical operations and neighborhood pr...
Contrast enhancement using various statistical operations and neighborhood pr...Contrast enhancement using various statistical operations and neighborhood pr...
Contrast enhancement using various statistical operations and neighborhood pr...sipij
 
Robust Super-Resolution by minimizing a Gaussian-weighted L2 error norm
Robust Super-Resolution by minimizing a Gaussian-weighted L2 error normRobust Super-Resolution by minimizing a Gaussian-weighted L2 error norm
Robust Super-Resolution by minimizing a Gaussian-weighted L2 error normTuan Q. Pham
 
Gi2429352937
Gi2429352937Gi2429352937
Gi2429352937IJMER
 
Image Restitution Using Non-Locally Centralized Sparse Representation Model
Image Restitution Using Non-Locally Centralized Sparse Representation ModelImage Restitution Using Non-Locally Centralized Sparse Representation Model
Image Restitution Using Non-Locally Centralized Sparse Representation ModelIJERA Editor
 
Quality and size assessment of quantized images using K-Means++ clustering
Quality and size assessment of quantized images using K-Means++ clusteringQuality and size assessment of quantized images using K-Means++ clustering
Quality and size assessment of quantized images using K-Means++ clusteringjournalBEEI
 
quantization and sampling presentation ppt
quantization and sampling presentation pptquantization and sampling presentation ppt
quantization and sampling presentation pptKNaveenKumarECE
 

Similar to Colorization with total variantion regularization (20)

D046022629
D046022629D046022629
D046022629
 
Iisrt zzz bhavyasri vanteddu
Iisrt zzz bhavyasri vantedduIisrt zzz bhavyasri vanteddu
Iisrt zzz bhavyasri vanteddu
 
Lect 02 second portion
Lect 02  second portionLect 02  second portion
Lect 02 second portion
 
Lec-3 DIP.pptx
Lec-3 DIP.pptxLec-3 DIP.pptx
Lec-3 DIP.pptx
 
A Four Directions Variational Method For Solving Image Processing Problems
A Four Directions Variational Method For Solving Image Processing ProblemsA Four Directions Variational Method For Solving Image Processing Problems
A Four Directions Variational Method For Solving Image Processing Problems
 
Presentation 1
Presentation 1Presentation 1
Presentation 1
 
Alternating direction-method-for-image-restoration
Alternating direction-method-for-image-restorationAlternating direction-method-for-image-restoration
Alternating direction-method-for-image-restoration
 
Lec_2_Digital Image Fundamentals.pdf
Lec_2_Digital Image Fundamentals.pdfLec_2_Digital Image Fundamentals.pdf
Lec_2_Digital Image Fundamentals.pdf
 
Fundamentals of image processing
Fundamentals of image processing  Fundamentals of image processing
Fundamentals of image processing
 
On image intensities, eigenfaces and LDA
On image intensities, eigenfaces and LDAOn image intensities, eigenfaces and LDA
On image intensities, eigenfaces and LDA
 
Contrast enhancement using various statistical operations and neighborhood pr...
Contrast enhancement using various statistical operations and neighborhood pr...Contrast enhancement using various statistical operations and neighborhood pr...
Contrast enhancement using various statistical operations and neighborhood pr...
 
Robust Super-Resolution by minimizing a Gaussian-weighted L2 error norm
Robust Super-Resolution by minimizing a Gaussian-weighted L2 error normRobust Super-Resolution by minimizing a Gaussian-weighted L2 error norm
Robust Super-Resolution by minimizing a Gaussian-weighted L2 error norm
 
It 603
It 603It 603
It 603
 
It 603
It 603It 603
It 603
 
Gi2429352937
Gi2429352937Gi2429352937
Gi2429352937
 
Image Restitution Using Non-Locally Centralized Sparse Representation Model
Image Restitution Using Non-Locally Centralized Sparse Representation ModelImage Restitution Using Non-Locally Centralized Sparse Representation Model
Image Restitution Using Non-Locally Centralized Sparse Representation Model
 
Quality and size assessment of quantized images using K-Means++ clustering
Quality and size assessment of quantized images using K-Means++ clusteringQuality and size assessment of quantized images using K-Means++ clustering
Quality and size assessment of quantized images using K-Means++ clustering
 
quantization and sampling presentation ppt
quantization and sampling presentation pptquantization and sampling presentation ppt
quantization and sampling presentation ppt
 
1873 1878
1873 18781873 1878
1873 1878
 
1873 1878
1873 18781873 1878
1873 1878
 

More from JÚLIO PEIXOTO

Análise de desempenho de arquiteturas de redes neurais para detecção de crise...
Análise de desempenho de arquiteturas de redes neurais para detecção de crise...Análise de desempenho de arquiteturas de redes neurais para detecção de crise...
Análise de desempenho de arquiteturas de redes neurais para detecção de crise...JÚLIO PEIXOTO
 
FORMA ANALÍTICA E MÉTODOS DAS DIFERENÇAS FINITAS APLICADO AO POTENCIAL DENTRO...
FORMA ANALÍTICA E MÉTODOS DAS DIFERENÇAS FINITAS APLICADO AO POTENCIAL DENTRO...FORMA ANALÍTICA E MÉTODOS DAS DIFERENÇAS FINITAS APLICADO AO POTENCIAL DENTRO...
FORMA ANALÍTICA E MÉTODOS DAS DIFERENÇAS FINITAS APLICADO AO POTENCIAL DENTRO...JÚLIO PEIXOTO
 
Aplicações supervisório
Aplicações supervisórioAplicações supervisório
Aplicações supervisórioJÚLIO PEIXOTO
 
Entradas E Saídas Digitais AnalóGicas
Entradas E Saídas Digitais AnalóGicasEntradas E Saídas Digitais AnalóGicas
Entradas E Saídas Digitais AnalóGicasJÚLIO PEIXOTO
 
Linguagens De ProgramaçãO Clp
Linguagens De ProgramaçãO ClpLinguagens De ProgramaçãO Clp
Linguagens De ProgramaçãO ClpJÚLIO PEIXOTO
 

More from JÚLIO PEIXOTO (8)

Análise de desempenho de arquiteturas de redes neurais para detecção de crise...
Análise de desempenho de arquiteturas de redes neurais para detecção de crise...Análise de desempenho de arquiteturas de redes neurais para detecção de crise...
Análise de desempenho de arquiteturas de redes neurais para detecção de crise...
 
FORMA ANALÍTICA E MÉTODOS DAS DIFERENÇAS FINITAS APLICADO AO POTENCIAL DENTRO...
FORMA ANALÍTICA E MÉTODOS DAS DIFERENÇAS FINITAS APLICADO AO POTENCIAL DENTRO...FORMA ANALÍTICA E MÉTODOS DAS DIFERENÇAS FINITAS APLICADO AO POTENCIAL DENTRO...
FORMA ANALÍTICA E MÉTODOS DAS DIFERENÇAS FINITAS APLICADO AO POTENCIAL DENTRO...
 
Aplicações supervisório
Aplicações supervisórioAplicações supervisório
Aplicações supervisório
 
Protocolo Can
Protocolo CanProtocolo Can
Protocolo Can
 
Clp Conceito
Clp ConceitoClp Conceito
Clp Conceito
 
Entradas E Saídas Digitais AnalóGicas
Entradas E Saídas Digitais AnalóGicasEntradas E Saídas Digitais AnalóGicas
Entradas E Saídas Digitais AnalóGicas
 
Sensores
SensoresSensores
Sensores
 
Linguagens De ProgramaçãO Clp
Linguagens De ProgramaçãO ClpLinguagens De ProgramaçãO Clp
Linguagens De ProgramaçãO Clp
 

Recently uploaded

analog-vs-digital-communication (concept of analog and digital).pptx
analog-vs-digital-communication (concept of analog and digital).pptxanalog-vs-digital-communication (concept of analog and digital).pptx
analog-vs-digital-communication (concept of analog and digital).pptxKarpagam Institute of Teechnology
 
Filters for Electromagnetic Compatibility Applications
Filters for Electromagnetic Compatibility ApplicationsFilters for Electromagnetic Compatibility Applications
Filters for Electromagnetic Compatibility ApplicationsMathias Magdowski
 
Introduction to Artificial Intelligence and History of AI
Introduction to Artificial Intelligence and History of AIIntroduction to Artificial Intelligence and History of AI
Introduction to Artificial Intelligence and History of AISheetal Jain
 
Linux Systems Programming: Semaphores, Shared Memory, and Message Queues
Linux Systems Programming: Semaphores, Shared Memory, and Message QueuesLinux Systems Programming: Semaphores, Shared Memory, and Message Queues
Linux Systems Programming: Semaphores, Shared Memory, and Message QueuesRashidFaridChishti
 
Seismic Hazard Assessment Software in Python by Prof. Dr. Costas Sachpazis
Seismic Hazard Assessment Software in Python by Prof. Dr. Costas SachpazisSeismic Hazard Assessment Software in Python by Prof. Dr. Costas Sachpazis
Seismic Hazard Assessment Software in Python by Prof. Dr. Costas SachpazisDr.Costas Sachpazis
 
Final DBMS Manual (2).pdf final lab manual
Final DBMS Manual (2).pdf final lab manualFinal DBMS Manual (2).pdf final lab manual
Final DBMS Manual (2).pdf final lab manualBalamuruganV28
 
5G and 6G refer to generations of mobile network technology, each representin...
5G and 6G refer to generations of mobile network technology, each representin...5G and 6G refer to generations of mobile network technology, each representin...
5G and 6G refer to generations of mobile network technology, each representin...archanaece3
 
UNIT-2 image enhancement.pdf Image Processing Unit 2 AKTU
UNIT-2 image enhancement.pdf Image Processing Unit 2 AKTUUNIT-2 image enhancement.pdf Image Processing Unit 2 AKTU
UNIT-2 image enhancement.pdf Image Processing Unit 2 AKTUankushspencer015
 
Involute of a circle,Square, pentagon,HexagonInvolute_Engineering Drawing.pdf
Involute of a circle,Square, pentagon,HexagonInvolute_Engineering Drawing.pdfInvolute of a circle,Square, pentagon,HexagonInvolute_Engineering Drawing.pdf
Involute of a circle,Square, pentagon,HexagonInvolute_Engineering Drawing.pdfJNTUA
 
Activity Planning: Objectives, Project Schedule, Network Planning Model. Time...
Activity Planning: Objectives, Project Schedule, Network Planning Model. Time...Activity Planning: Objectives, Project Schedule, Network Planning Model. Time...
Activity Planning: Objectives, Project Schedule, Network Planning Model. Time...Lovely Professional University
 
Vip ℂall Girls Karkardooma Phone No 9999965857 High Profile ℂall Girl Delhi N...
Vip ℂall Girls Karkardooma Phone No 9999965857 High Profile ℂall Girl Delhi N...Vip ℂall Girls Karkardooma Phone No 9999965857 High Profile ℂall Girl Delhi N...
Vip ℂall Girls Karkardooma Phone No 9999965857 High Profile ℂall Girl Delhi N...jiyav969
 
8th International Conference on Soft Computing, Mathematics and Control (SMC ...
8th International Conference on Soft Computing, Mathematics and Control (SMC ...8th International Conference on Soft Computing, Mathematics and Control (SMC ...
8th International Conference on Soft Computing, Mathematics and Control (SMC ...josephjonse
 
Operating System chapter 9 (Virtual Memory)
Operating System chapter 9 (Virtual Memory)Operating System chapter 9 (Virtual Memory)
Operating System chapter 9 (Virtual Memory)NareenAsad
 
Electrical shop management system project report.pdf
Electrical shop management system project report.pdfElectrical shop management system project report.pdf
Electrical shop management system project report.pdfKamal Acharya
 
Online crime reporting system project.pdf
Online crime reporting system project.pdfOnline crime reporting system project.pdf
Online crime reporting system project.pdfKamal Acharya
 
Multivibrator and its types defination and usges.pptx
Multivibrator and its types defination and usges.pptxMultivibrator and its types defination and usges.pptx
Multivibrator and its types defination and usges.pptxalijaker017
 
The battle for RAG, explore the pros and cons of using KnowledgeGraphs and Ve...
The battle for RAG, explore the pros and cons of using KnowledgeGraphs and Ve...The battle for RAG, explore the pros and cons of using KnowledgeGraphs and Ve...
The battle for RAG, explore the pros and cons of using KnowledgeGraphs and Ve...Roi Lipman
 
Diploma Engineering Drawing Qp-2024 Ece .pdf
Diploma Engineering Drawing Qp-2024 Ece .pdfDiploma Engineering Drawing Qp-2024 Ece .pdf
Diploma Engineering Drawing Qp-2024 Ece .pdfJNTUA
 
handbook on reinforce concrete and detailing
handbook on reinforce concrete and detailinghandbook on reinforce concrete and detailing
handbook on reinforce concrete and detailingAshishSingh1301
 
"United Nations Park" Site Visit Report.
"United Nations Park" Site  Visit Report."United Nations Park" Site  Visit Report.
"United Nations Park" Site Visit Report.MdManikurRahman
 

Recently uploaded (20)

analog-vs-digital-communication (concept of analog and digital).pptx
analog-vs-digital-communication (concept of analog and digital).pptxanalog-vs-digital-communication (concept of analog and digital).pptx
analog-vs-digital-communication (concept of analog and digital).pptx
 
Filters for Electromagnetic Compatibility Applications
Filters for Electromagnetic Compatibility ApplicationsFilters for Electromagnetic Compatibility Applications
Filters for Electromagnetic Compatibility Applications
 
Introduction to Artificial Intelligence and History of AI
Introduction to Artificial Intelligence and History of AIIntroduction to Artificial Intelligence and History of AI
Introduction to Artificial Intelligence and History of AI
 
Linux Systems Programming: Semaphores, Shared Memory, and Message Queues
Linux Systems Programming: Semaphores, Shared Memory, and Message QueuesLinux Systems Programming: Semaphores, Shared Memory, and Message Queues
Linux Systems Programming: Semaphores, Shared Memory, and Message Queues
 
Seismic Hazard Assessment Software in Python by Prof. Dr. Costas Sachpazis
Seismic Hazard Assessment Software in Python by Prof. Dr. Costas SachpazisSeismic Hazard Assessment Software in Python by Prof. Dr. Costas Sachpazis
Seismic Hazard Assessment Software in Python by Prof. Dr. Costas Sachpazis
 
Final DBMS Manual (2).pdf final lab manual
Final DBMS Manual (2).pdf final lab manualFinal DBMS Manual (2).pdf final lab manual
Final DBMS Manual (2).pdf final lab manual
 
5G and 6G refer to generations of mobile network technology, each representin...
5G and 6G refer to generations of mobile network technology, each representin...5G and 6G refer to generations of mobile network technology, each representin...
5G and 6G refer to generations of mobile network technology, each representin...
 
UNIT-2 image enhancement.pdf Image Processing Unit 2 AKTU
UNIT-2 image enhancement.pdf Image Processing Unit 2 AKTUUNIT-2 image enhancement.pdf Image Processing Unit 2 AKTU
UNIT-2 image enhancement.pdf Image Processing Unit 2 AKTU
 
Involute of a circle,Square, pentagon,HexagonInvolute_Engineering Drawing.pdf
Involute of a circle,Square, pentagon,HexagonInvolute_Engineering Drawing.pdfInvolute of a circle,Square, pentagon,HexagonInvolute_Engineering Drawing.pdf
Involute of a circle,Square, pentagon,HexagonInvolute_Engineering Drawing.pdf
 
Activity Planning: Objectives, Project Schedule, Network Planning Model. Time...
Activity Planning: Objectives, Project Schedule, Network Planning Model. Time...Activity Planning: Objectives, Project Schedule, Network Planning Model. Time...
Activity Planning: Objectives, Project Schedule, Network Planning Model. Time...
 
Vip ℂall Girls Karkardooma Phone No 9999965857 High Profile ℂall Girl Delhi N...
Vip ℂall Girls Karkardooma Phone No 9999965857 High Profile ℂall Girl Delhi N...Vip ℂall Girls Karkardooma Phone No 9999965857 High Profile ℂall Girl Delhi N...
Vip ℂall Girls Karkardooma Phone No 9999965857 High Profile ℂall Girl Delhi N...
 
8th International Conference on Soft Computing, Mathematics and Control (SMC ...
8th International Conference on Soft Computing, Mathematics and Control (SMC ...8th International Conference on Soft Computing, Mathematics and Control (SMC ...
8th International Conference on Soft Computing, Mathematics and Control (SMC ...
 
Operating System chapter 9 (Virtual Memory)
Operating System chapter 9 (Virtual Memory)Operating System chapter 9 (Virtual Memory)
Operating System chapter 9 (Virtual Memory)
 
Electrical shop management system project report.pdf
Electrical shop management system project report.pdfElectrical shop management system project report.pdf
Electrical shop management system project report.pdf
 
Online crime reporting system project.pdf
Online crime reporting system project.pdfOnline crime reporting system project.pdf
Online crime reporting system project.pdf
 
Multivibrator and its types defination and usges.pptx
Multivibrator and its types defination and usges.pptxMultivibrator and its types defination and usges.pptx
Multivibrator and its types defination and usges.pptx
 
The battle for RAG, explore the pros and cons of using KnowledgeGraphs and Ve...
The battle for RAG, explore the pros and cons of using KnowledgeGraphs and Ve...The battle for RAG, explore the pros and cons of using KnowledgeGraphs and Ve...
The battle for RAG, explore the pros and cons of using KnowledgeGraphs and Ve...
 
Diploma Engineering Drawing Qp-2024 Ece .pdf
Diploma Engineering Drawing Qp-2024 Ece .pdfDiploma Engineering Drawing Qp-2024 Ece .pdf
Diploma Engineering Drawing Qp-2024 Ece .pdf
 
handbook on reinforce concrete and detailing
handbook on reinforce concrete and detailinghandbook on reinforce concrete and detailing
handbook on reinforce concrete and detailing
 
"United Nations Park" Site Visit Report.
"United Nations Park" Site  Visit Report."United Nations Park" Site  Visit Report.
"United Nations Park" Site Visit Report.
 

Colorization with total variantion regularization

  • 1. Colorization with total variation regularization J´ulio Peixoto da Silva J´unior Fazal-E-Asim Weskley Vinicius Fernandes Mauricio Yosbel Rodrigues Ortega Departamento de Teleinform´atica - UFC Non-Linear Optimization July 19, 2017 DETI-UFC Non-Linear Optimization July 19, 2017 1 / 25
  • 2. Outlines 1 Problem Statement 2 Total Variation Regularization 3 Examples using Boyd’s method (CVX) 4 TV denoising solution. 5 Examples using TV denoising Solution (CVX) 6 Colorization by propagation method 7 Examples using propagation method 8 Comparison and Analysis 9 Conclusions DETI-UFC Non-Linear Optimization July 19, 2017 2 / 25
  • 3. Problem Statement A color image (mxn) is represented as three matrices of intensities R,G,B ∈ Rmxn with entries in [0,1] representing the red, green, and blue pixel intensities, respectively. A color image is converted to a monochrome image, represented as one matrix M ∈ Rmxn using M = 0.299R + 0.587G + 0.114B (1) DETI-UFC Non-Linear Optimization July 19, 2017 3 / 25
  • 4. Cont’d In colorization, we are given M, the monochrome version of an image, and the color values of some of the pixels. We are to guess its color version,i.e.,the matrices R,G,B. We have to minimize the total variation function, defined as: tv(R, G, B) = m−1 i=1 n−1 j=1         Rij − Ri,j+1 Gij − Gi,j+1 Bij − Bi,j+1 Rij − Ri+1,j Gij − Gi+1,j Bij − Bi+1,j         2 (2) DETI-UFC Non-Linear Optimization July 19, 2017 4 / 25
  • 5. Cont’d Subject to consistency with the given monochrome image M, the knows ranges of the entries of (R,G,B) (i.e.,∈ [0,1]), and the given color entries. Monochrome version of the image, M, along with vectors of known color intensities is given to us. The tv function, invoked as tv(R,G,B), gives the total variation. Report your optimal objective value and if you have access to a color printer, attach your reconstructed image. If you don’t have access to a color printer, it’s OK to just give the optimal objective value. DETI-UFC Non-Linear Optimization July 19, 2017 5 / 25
  • 6. Cont’d Convex Problem Every norms on Rn is convex! minimize tv(R, G, B) subject to: R(knom ind) == R know G(knom ind) == G know B(knom ind) == B know 0.229R + 0.587G + 0.114B == M R, G and B ∈ [0, 1] (3) DETI-UFC Non-Linear Optimization July 19, 2017 6 / 25
  • 7. Examples using Boyd’s method (CVX) Read the image Read the image flower.png (50x50) Grayscale Image Apply M = 0.229R + 0.587G + 0.114 ∗ B to obtain grayscale image Create Damage Image known ind = find(rand(m, n) >= 0.90) Solver Using CVX for solve the problem Reconstructed Image Using the results of CVX for reconstructed the image. DETI-UFC Non-Linear Optimization July 19, 2017 7 / 25
  • 8. Results with CVX - Exampe I (a) Original (b) Monochrome (c) Given (d) Reconstructed DETI-UFC Non-Linear Optimization July 19, 2017 8 / 25
  • 9. Results with CVX - Exampe II (e) Original (f) Monochrome (g) Given (h) Reconstructed DETI-UFC Non-Linear Optimization July 19, 2017 9 / 25
  • 10. TV denoising solution minimize ||X − Y || + λ · tv(R, G, B) subject to: R(knom ind) == R know G(knom ind) == G know B(knom ind) == B know 0.229R + 0.587G + 0.114B == M R, G and B ∈ [0, 1] (4) Definition of variables where X is the concatenation of R, G and B and Y is the monochromatic image, in vector form. λ is a positive scale factor. DETI-UFC Non-Linear Optimization July 19, 2017 10 / 25
  • 11. Examples using TV denoising solution - Lambda = 0.1 (i) Original (j) Monochrome (k) Given (l) Reconstructed DETI-UFC Non-Linear Optimization July 19, 2017 11 / 25
  • 12. Examples using TV denoising solution - Lambda = 1 (m) Original (n) Monochrome (o) Given (p) Reconstructed DETI-UFC Non-Linear Optimization July 19, 2017 12 / 25
  • 13. Examples using TV denoising solution - Lambda = 10 (q) Original (r) Monochrome (s) Given (t) Reconstructed DETI-UFC Non-Linear Optimization July 19, 2017 13 / 25
  • 14. Colorization by Propagation YUV color space, is a model commonly used in video, where Y is the monochromatic luminance channel (intensity), while U and V are the chrominance channels, encoding the color. The method called Propagation using Optimization, is given as input an intensity volume Y (x; y; t) and outputs two color volumes U (x; y; t) and V (x; y; t). Figure 1: YUV color space Thus, if Y(r) is the intensity of a particular pixel, then we wish to impose the constraint that two neighboring pixels r, s should have similar colors if their intensities are similar. DETI-UFC Non-Linear Optimization July 19, 2017 14 / 25
  • 15. Cont’d Neighboring pixels with similar intensities should have similar colors, so, the colors can be propagated to all pixels from known color pixel. Figure 2: Neighboring pixels. DETI-UFC Non-Linear Optimization July 19, 2017 15 / 25
  • 16. Cont’d The problem results in minimize the difference between the color U(r) at pixel r and the affinity-weighted average of the colors at neighboring pixels (s): Objective Function minimize J(U) = r U(r) − s∈N(r) wrsU(s) (5) where wrs is a affinity-weighting function that sums to one, large when Y(r) is similar to Y (s), and small when the two intensities are different. The notation s ∈ N(r) denotes the fact that r and s are neighboring pixels. DETI-UFC Non-Linear Optimization July 19, 2017 16 / 25
  • 17. Cont’d There are in the literature different types of affinity-weighting functions. The simplest one is commonly used by image segmentation algorithms and is based on the squared difference between the two intensities: wrs ∝ exp − (Y (r) − Y (s))2 2σ2 r (6) σ proportional to local variance. Figure 3: Affinity-weighting functions. DETI-UFC Non-Linear Optimization July 19, 2017 17 / 25
  • 18. Results with Colorization by Propagation - Example I (a) Original (b) Monochrome (c) Given (d) Reconstructed DETI-UFC Non-Linear Optimization July 19, 2017 18 / 25
  • 19. Results with Colorization by Propagation - Exampe II (e) Original (f) Monochrome (g) Given (h) Reconstructed DETI-UFC Non-Linear Optimization July 19, 2017 19 / 25
  • 20. Metrics used for Comparison Mean Square Error (MSE) MSE = 1 MN M y=1 N x=1 [I(x, y) − ˜I(x, y)]2 (7) Peak Signal to Noise Ratio (PSNR) PSNR = 20 ∗ log10 255 √ MSE (8) DETI-UFC Non-Linear Optimization July 19, 2017 20 / 25
  • 21. Comparison of different methods TV function Value of Original Image 411.9606 Table 1: Results example I - Boyd Julia Propagation Denoising Solver SDPT31, SeDuMi2 SCS3 - SDPT3 TV function 341.47 341.73 383.43 342.24 Time(s) 2.62 7.58 0.3067 4.20 MSE 0.0058 0.0057 0.0022 0.0056 PSNR(dB) 70.49 70.57 74.70 70.64 1 A MATLAB software for semidefinite-quadratic-linear programming. 2 Optimization over symmetric cones. 3 Solver for semidefinite, second-order and exponential cone programming. DETI-UFC Non-Linear Optimization July 19, 2017 21 / 25
  • 22. Cont’d ... TV Function Value of Original Image 28560 Table 2: Results example II - Matlab Julia Propagation Denoising∗ Solver SDPT3, SeDuMi SCS4 - SDTP3 TV function 25539 25150 28126 - Time(s) 746,3 3547.13 38,0 - MSE 0.0015 0.0018 0.000159 - PSNR(dB) 76.36 75.5781 86.1168 - 4 Solution with intermittent behavior, only one time solved with success. DETI-UFC Non-Linear Optimization July 19, 2017 22 / 25
  • 23. Conclusions We solved the problem using Boyd’s method using MATLAB and JULIA. We also used TV denoising and colorization by propagation methods to solve the same problem in MATLAB. The propagation method have the best solution as compared to other methods. The Boyd’s method shows least performance as compared to other method.. DETI-UFC Non-Linear Optimization July 19, 2017 23 / 25
  • 24. References 1 S. Boyd and L. Vandenberghe. Convex Optimization. Cambridge University Press, 2004. 2 S. Boyd, Jenny Hong el al. Convex Optimization in Julia. HPTCDL November 16-21, 2014, New Orleans, Louisiana, USA 3 Karanveer Mohan, Madeleine Udell, David Zeng, Jenny Hong. Convex.jl Documentation. Jun 24, 2017. 4 Nonlinear total variation based noise removal algorithms, Rudin, L. I.; Osher, S.; Fatemi, E. (1992). Physica D 60: 259–268. 5 A generalized vector-valued total variation algorithm., Rodriguez Paul, and Brendt Wohlberg. Image Processing (ICIP), 2009 16th IEEE International Conference on. IEEE, 2009. 6 R´emi Flamary. Avaliable at: http://remi.flamary.com/demos/proxtv.html [Acessed - 07/16] 7 LEVIN, A., LISCHINSKI, D., AND WEISS, Y. 2004. Colorization using optimization. ACM Trans. Graph. 23, 3 (Aug.), 689–694. DETI-UFC Non-Linear Optimization July 19, 2017 24 / 25
  • 25. The End Thank you. Gracias. Obrigado DETI-UFC Non-Linear Optimization July 19, 2017 25 / 25