This presentation shows the two techniques to colorize the gray scale images. (1) Optimization based colorization (2) Pseudocoloring

1. Department of Computer Science
Gujarat University
IMAGE COLORIZATION
Presented By:
- Fadia Pankti
Course:
- M.Sc.(AI & ML) – II
Under the Guidance of:
- Dr. Jyoti Pareek

2. T a b l e O f C o n t e n t s
 1. Introduction
1.1 Objective
1.2 Scope
1.3 What is image colorization?
 2. Application of colorization
 3. Literature Survey (Image colorization techniques)
 4. Approach (Optimization & Pseudocoloring)
 5. Testing & Results
 6. Conclusion & Future Directions
 7. References
1

3. O B J E C T I V E
1. Adding color to monochrome images
2. Improve visual appeal of illustrations.
2
Index

4. S C O P E
 Recent scope:
 Initially gray scale images are taken of different domain.
 Scribble based colorization .
 Apply pseudo color to medical and satellite images using
colormaps.
3
Index

5. W h a t I s I m a g e C o l o r i z a t i o
n ?
 Image colorization is the process of taking an input grayscale or (black and
white) image and then producing an output colorized image that represents the
semantic colors and tones of the input .
 Color image contain more information than gray scale image. And it is more useful
to extract information from color image and it is visually appealing to viewers.
4
Index

6. C o n t i n u e . . .
 Color image consists of three dimensional information about the color of image .
RED,BLUE, GREEN . Where as gray scale image consist of luminance and hence it is
one
dimensional.
 Converting color image to grey scale image means we are dropping information
about color.
 It is easy to convert color image to grey scale image but reverse is not easy.
Because there can be numerous colors which represents one grey level, but in
5
Index

7. C o l o r i z a t i o n A p p l i c a t i o n s
 Airport System
 Satellite Imaging
 Old photos and films
 Medical Imaging
6
Index

8. I m a g e C o l o r i z a t i o n
T e c h n i q u e s
Colorization technique can be classified under four categories:
1. Manual coloring
2. Semiautomatic coloring
3. Advance semiautomatic coloring
4. Fully automatic coloring
7
Index

9.  In manual coloring, a very extensive human effort is required. we have to color different
parts of image by our perception of the colors of gray image. We have to select each pixel
of the gray image to get the job done.
 Mostly image editing software like paint or adobe Photoshop are used for converting gray
image to colored image.
 Disadvantage:
 The main drawback of this method is that a segmentation task is done completely manually
and this process is very time consuming and tedious.
 Requires human vision of proposed colors of gray pixel.
8
1. M a n u a l C o l o r i n g
Index

10. E x a m p l e
9
Photoshop editor
Index

11.  In semi-automatic coloring, user provides some colored scribbles and then the color is
propagated over the image based on an optimization framework.
 Though, this method is efficient but it is time consuming. It is more useful in medical
image application like X-Ray , CT-Scan and MRI images.
 Disadvantage:
 Such methods require users to provide a considerable number of scribbles on the
grayscale image, which is time-consuming and requires expertise .
 Moreover, it is almost impossible to add such markings to large images.
2. S e m i a u t o m a t i c C o l o r i n
g
10
Index

12. E x a m p l e
Original gray scale
Image
Marked (Scribbled)
Image
Color Image
11
Index

13.  Unlike manual coloring and semi-automatic coloring, advance automatic coloring technique
removes the user intervention to a large extent .
 A reference image is considered and it transfers the color information from a similar
reference image to the input grayscale image rather than obtaining chromatic values from
the user, thereby reducing the burden on users.
 Disadvantage:
 Feature matching is critical to the quality of the results so, satisfactory results cannot be
obtained if feature matching is not performed correctly.
 Moreover, the procedure is very sensitive to image brightness and contrast, whereas real
aerial images always include large areas of shadow and low contrast.
3. A d v a n c e S e m i a u t o m a t i c
C o l o r i n g
12
Index

14. E x a m p l e
Colorized ImageReference Colored ImageGray Image
13
Index

15.  An alternative approach is to employ a large number of training images , which is a recent
example of deep learning.
 A large database of color images comprising all kinds of objects is used for training the
neural networks. the source image is retrieved automatically from a database of images by
some matching procedure.
 The trained model can then be used to efficiently transfer the color information to the
grayscale image images. In these methods user takes no effort in source image selection as
well as in color transfer process.
 Disadvantage:
 This approach is computationally expensive, and the training is significantly slow.
4. F u l l y A u t o m a t i c C o l o r i n
g
14
Index

16. E x a m p l e
15
Index

17. O p t i m i z a t i o n B a s e d
c o l o r i z a t i o n
 Introduction:
 Levin et al. solved an optimization problem that minimizes a quadratic cost function of
the difference of color between a pixel and it’s weighted average neighborhood
 The algorithm propagates the color information from pixels colored by hand(scribbles) or
any software to the rest of the image
 neighboring pixels in space-time that have similar intensities should have similar colors.
 These constraints are formulated as a least-squares optimization problem that
automatically propagates the scribbled colors to produce a completely colorized image.
16
Index

18. W o r k F l o w
1) Convert both the original gray-scale image and the marked image (marked with color
scribbles for a few pixels by the user) to from RGB to YUV / YIQ color space.
 The algorithm works in YUV color space where the Y component represents
brightness, and U,V components represent the chrominance information of
pixel.
 The algorithm is given as input an intensity value Y(x; y; t) and outputs two color values
U(x; y; t) and V(x; y; t).
 To simplify notation we will use letters r and s to denote (x; y; t) triplets.
2) Compute the difference image from the marked and the gray-scale image.
17
IndexReference: [9]

19. 18
 As mentioned in the introduction, we wish to impose the constraint that two
neighboring pixels r; s should have similar colors if their intensities are similar.
 Assuming g a gray scale image set the luminance channel of the colored image
as Y = g and the chrominance components u and v are computed by minimizing:
Reference: [8] Index

20. 19
3) We need to compute the weight matrix W that depends on the similarities in
the neighbor intensities for a pixel from the original gray-scale image.
 Wrs is a weighting function that sums to one, large when Y(r) has similar intensity to
Y(s), and small when the two intensities are different. Y(r), Y(s) are luminance value of
r and s, and σr represents the variance of the luminance in a window around r.
Index

21. 20
5) However the W matrix is going to be very huge and sparse, hence sparse-matrix based
implementations is used to obtain an efficient solution.
6) Once W is computed obtain the least-square solution, by computing the pseudo-inverse.
7) Once the solution of the optimization problem is obtained in YUV / YIQ space, it needs to
be
converted back to RGB. The following formula is used for conversion.
Index
4) The next step is to solve the system of linear equations :
 it assumes that the colour at a pixel U(r) is a linear function of the intensity
Y(r):U(r) =(ai*Y(r)) +bi and the linear coefficients ai, bi are the same for all pixels in a
small neighbourhood around r.

22. R e s u l t
Original gray scale
Image
Marked (Scribbled)
Image
Color Image
21
Index

23. P s e u d o c o l o r i n g
 Definition:
 A pseudo color image is derived from a gray scale image by mapping each intensity
value to a color according to a table or function.
Pseudo color=False color
 In some cases, there is no color concept for gray scale image, but we can assign false
color to image.
 Transformation of a gray scale image into pseudo color image helps in better
visualization of the image.
22
Index

24. W h a t I s C o l o r m a p ?
 Color mapping is a function that maps (transforms) the colors of one (source) image
to the colors of another (target) image.
 A color mapping may be referred to as the algorithm that results in the mapping
function or the algorithm that transforms the image colors.
 Map the position of the values in colormap to the intensity or the pixel value of the
gray scale image and replace it with the corresponding value from the colormap.
23
Index

25. C o n t i n u e . . .
 The main idea behind pseudo color transformation is to perform three independent
transformation (RED,GREEN and BLUE) on the grayscale or intensity image and
map the corresponding intensity value in the image to the result obtained.
24
Index

26. 25
RED GREEN BLUE
E x a m p l e
Gray Intensity matrix Defined Colormap
First column represents RED , second
column represent Green and last column
represents Blue
Reference : https://www.imageeprocessing.com/2016/03/gray-scale-to-pseudo-color.html Index

27. 26
Pre allocate matrix
Map the position of values in colormap to
pixel value of the gray scale image and
replace it with corresponding value from
color map
Update matrix
(1) (2)
(4)
(3)
Index

28. C o n t i n u e . . .
 we have implemented OpenCV’s predefined colormaps to pseudo color / false
color a grayscale image of Medical and Satellite.
27
IndexReference: [10]

29. R e s u l t O f M e d i c a l I m a g e s
28
Original gray scale Image
Color image using Different Colormap
Index

30. Original gray scale Image
Color image using Different Colormap
29
R e s u l t O f S a t e l l i t e I m a g e s
Index

31. C o n c l u s i o n & F u t u r e
D i r e c t i o n s
 There is no need to explicitly delineate the exact boundaries of objects.
 Instead, we can color a small number of pixels in selected frames and the algorithm propagates these
colors in a manner that respects intensity boundaries. We have shown that excellent colorizations can
be obtained with a surprisingly small amount of user effort.
 Also we have seen that pseudo color gives us very good representations of colors although false color
sacrifices natural color rendition in order to ease the detection of features that are not readily
discernible otherwise but for the images of medical and satellite where natural color is difficult to
predict pseudo color provides good results.
30
Index

32.  Future Directions:
 In future work, we will build on this equivalence and import advances in image
segmentation into the problem of colorization.
 Additionally, we plan to explore alternative color spaces and propagation
schemes.
 We will explore other algorithm like reference based and neural networks to
colorize image.
31
C o n t i n u e . . .
Index

33. R e f e r e n c e s
1. https://projet.liris.cnrs.fr/imagine/pub/proceedings/ICIP-2014/Papers/1569912999.pdf
2. https://www.cs.huji.ac.il/~yweiss/Colorization/
3. https://www.cs.unc.edu/~lazebnik/research/fall08/lec06_colorization.pdf
4. https://sites.google.com/site/fanyangspace111/image-colorization-by-scribbling
5. https://www.scirp.org/journal/paperinformation.aspx?paperid=87936#ref6
6. https://www.imageeprocessing.com/2016/03/gray-scale-to-pseudo-color.html
7. https://www.ft.unicamp.br/docentes/magic/khoros/html-dip/c4/s9/front-page.html
8. https://www.cs.tau.ac.il/~dcor/Graphics/pdf.slides/colorization.pdf
9. https://sandipanweb.wordpress.com/2018/01/27/image-colorization-using-optimization-in-python/
10. https://www.learnopencv.com/applycolormap-for-pseudocoloring-in-opencv-c-python/
32
Index

34. T h a n k Y o u . . .
30