Gray Image Coloring Using Texture Similarity Measures

3,294 views

Published on

Master Discussion
"Gray Image Coloring Using Texture Similarity Measures"
Faculty of Computers and Information - Menoufia University 2007

Published in: Technology, Business
0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total views
3,294
On SlideShare
0
From Embeds
0
Number of Embeds
3
Actions
Shares
0
Downloads
61
Comments
0
Likes
0
Embeds 0
No embeds

No notes for slide

Gray Image Coloring Using Texture Similarity Measures

  1. 1. Gray Image Coloring Using Texture Similarity Measures by E. Noura Abd El-Moez Semary Thesis Submitted in accordance with the requirements of The University of Monofiya for the degree of Master of Computers and Information 1 ( Information Technology )
  2. 2. Thesis summary on: Gray Image Coloring Using Texture Similarity Measures Prof. Mohiy Prof. Nabil Dr.Waiel .S.Supervised by: .M.Hadhoud .A.Ismail Al-Kilani Presented by E. Noura Abd El-Moez Semary For Master degree in Computers and Information IT department, Faculty of Computers and information, Menofia University
  3. 3. ‫:ملخص رسالة بعنوان‬ ‫تلوين الصور الرمادية بإستخدام معايير تشابه‬ ‫النسجة‬‫د. وائل شوقي‬ ‫أ.د. نبيل عبد الواحد‬ ‫أ.د. محي محمد‬ ‫:تحت إشراف‬ ‫الكيلني‬ ‫إسماعيل‬ ‫هدهود‬ ‫:مقدم من‬ ‫م . نورا عبد المعز السباعي سمري‬ ‫للحصول على درجة الماجستير في الحاسبات والمعلومات‬ ‫قسم تكنولوجيا المعلومات - كلية الحاسبات و المعلومات - جامعة المنوفية‬
  4. 4. Outlines Outlines Introduction Introduction  Automatic  Automatic coloring in the literature coloring in the literature TRICS ‘Texture Recognition based Image‘Texture TRICS  Recognition Coloring System’ based Image Coloring Results System’ Results  Conclusion Conclusion  Future work  Future work 4
  5. 5. IntroductionGray image principles Outlines  Introduction  Automatic coloring in the literature  TRICS ‘Texture Recognition based Image Coloring System’ Gray values  Results  Conclusion  Future work . . . . . . . . . . . 0 50 100 150 200 250 255 5
  6. 6. IntroductionGray image principles Image Size Possible No. Actually No. Total size Image (pixel) colors used colors (byte) px 120× 170 16,777,216 15,998 60.0 kb 170 ×120 px 256 246 21.2 kb 6
  7. 7. IntroductionColoring ProblemThere are two definitions to describe the gray value as an equation of the three basic components of RGB color model (red, green, blue):1: Intensity (most common used) Gray = (Red + Green + Blue) /32: Luminance (NTSC standard for luminance) Gray = (0.299 × Red) + (0.587 × Green) + (0.114 × Blue) RGB Color R, G, B values Gray value Gray Color 87 ,150 ,100 128 147, 87, 149 128 149, 147, 87 128 THERE IS NO MATHEMATICAL FORMULA TO CONVERT FROM GRAY TO RGB 7
  8. 8. IntroductionColoring Problem HSL Color wheel Grayed Color Wheel Similar Gray Values Similar Gray Values 8
  9. 9. IntroductionColoring Types1 . Hand coloring  Adobe Photoshop and Paintshop Pro  Layers  Changing Hue  BlackMagic, photo colorization software, version 2.8, 2003 9
  10. 10. IntroductionColoring Types2 . Semi automatic coloring  Pseudocoloring is a common example for semi automatic coloring technique 10
  11. 11. IntroductionColoring Types Outlines3 . Automatic coloring  Introduction  Automatic i. Transformational coloring coloring in the literature ii. Matched image coloring  TRICS ‘Texture Recognition iii. User selected coloring based Image Coloring System’  Results  Conclusion  Future work 11
  12. 12. Automatic coloring in the literature1. Transformational Coloring Outlines A transformation function Tk is applied on Introduction  Automatic  the intensity value of each pixel coloring in the Ig(i,j) literature resulting in the chromatic value Ick(i,j) for TRICS ‘Texture  Recognition channel k based Image Coloring System’  Results Ic k (i, j ) = Tk [ Ig (i, j )]  Conclusion  Future work 12
  13. 13. Automatic coloring in the literature1. Transformational Coloring Al-Gindy et al * system. × Results have unnatural look* A. N. Al-Gindy, H. Al Ahmad, R. A. Abd Alhameed, M. S. Abou Naaj and P. S. Excell ’Frequency Domain Technique For Colouring Gray Level Images’ 2004 found inwww.abhath.org/html/modules/pnAbhath/download.php?fid=32 13
  14. 14. Automatic coloring in the literature 2. Matched image coloring  The most similar pixel color is transferred to the corresponding gray one by the color transfer technique proposed by E.Reinhard*; l α β* Reinhard, E. Ashikhmin, M., Gooch B. And Shirley, P., Color Transfer between Images, IEEE ComputerGraphics and Applications, September/October 2001, 34-40 14
  15. 15. Automatic coloring in the literature 2. Matched image coloring  “Global matching procedure” of T. Welsh et al*  “Local color transfer” of Y. Tai et al**. × All these algorithms fail, when different colored regions have similar intensities* T. Welsh, M. Ashikhmin, K. Mueller. “Transferring color to greyscale images.” In Proceedings of the 29th Annual Conference on Computer Graphics and interactive Techniques, pp 277–280, 2002** Y. Tai, J. Jia, C. Tang ‘Local Color Transfer via Probabilistic Segmentation by Expectation- maximization‘, 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR05), Volume 1, pp. 747-754, 2005 15
  16. 16. Automatic coloring in the literature2. Matched image coloring  Welsh et al proposed also another technique to improve the coloring results when the matching results are not satisfying. It was achieved by asking users to identify and associate small rectangles, called “swatches” in both the source and destination images to indicate how certain key colors should be transferred 16
  17. 17. Automatic coloring in the literature3. User selection coloring Outlines Introduction  Automatic coloring in the literature  TRICS ‘Texture Recognition based Image User selection coloring gives high quality colors Coloring× User dependent color quality System’  Results× Time-consuming  Conclusion× Colorization must be fully recomputed for any slight change Future work  in the initial marked pixels* A. Levin, D. Lischinski, Y. Weiss. “Colorization using optimization.” ACM Transactions on Graphics,Volume 23, Issue 3, pp.689–694, 2004 17
  18. 18. TRICS System Research Objectives Outlines To simulate the human vision in coloring Introduction  process Automatic  coloring in the literature To be fully automatic coloring system TRICS ‘Texture  Recognition To spend so little execution time as possible based Image Coloring as a basic requirement for video coloring. System’  Results  Conclusion  Future work 18
  19. 19. TRICS SystemStructure Gray image 1 2 A Segmentation Features extraction Segmentation (Joint, wavelets, laws,…) (Mean Shift, K-Mean, FCM,..) Segmented image, Clusters 3 4 Classification B Features extraction Classification (Co-occurrence, Tamura, Wavelets energies) (K-NN classifier,..) Samples features Class labels Classes Hues Database 5 6 7 Coloring C Convert image to Set Hue, Saturation, and Convert to HSV channels Brightness RGB 19 Colored image
  20. 20. TRICS SystemStructure 1. Segmentation Stage Feature extraction: (Pixel based ) 1. pixel position 2. pixel intensity 3. texture features  wavelets coefficients  Laws kernels coefficients. 20
  21. 21. TRICS SystemStructure 1. Segmentation Stage1. Wavelets coefficients × Quarter the image size.  Up sampling  Upper level construction 21
  22. 22. TRICS SystemStructure 1. Segmentation Stage Up sampling Upper level construction 22
  23. 23. TRICS SystemStructure 1. Segmentation Stage2. Laws Kernels :Level L5 = [ 1 4 6 4 1]Edge E5 = [ -1 –2 0 2 1]Spot S5 = [ -1 0 2 0 –1]Wave W5= [ -1 2 0 –2 1]Ripple R5 = [ 1 –4 6 –4 1] -1 -4 -6 -4 -1 0 0 0 0 0L5S5’ = 2 8 12 8 2 0 0 0 0 0 -1 -4 -6 -4 -1 23
  24. 24. TRICS SystemStructure 1. Segmentation Stage Segmentation technique:  Mean Shift *  K-mean (Fast k-mean) **  Adaptive Fast k-mean* D. Comaniciu and P. Meer. ‘Mean shift: A robust approach toward feature spaceanalysis.’ PAMI, 24(5):603–619, May 2002** C.Elkan, ‘Using the triangle inequality to accelerate k-Means.’ In Proc. of ICML2003. pp 147--153 24
  25. 25. Mean Shift Region of interest Center of mass Mean Shift vector Objective : Find the densest region 25
  26. 26. Mean Shift Region of interest Center of mass Mean Shift vector Objective : Find the densest region 26
  27. 27. Mean Shift Region of interest Center of mass Mean Shift vector Objective : Find the densest region 27
  28. 28. Mean Shift Region of interest Center of mass Mean Shift vector Objective : Find the densest region 28
  29. 29. Mean Shift Region of interest Center of mass Mean Shift vector Objective : Find the densest region 29
  30. 30. Mean Shift Region of interest Center of mass Mean Shift vector Objective : Find the densest region 30
  31. 31. Mean Shift Region of interest Center of mass Objective : Find the densest region 31
  32. 32. TRICS SystemStructure 1. Segmentation Stage1. Mean Shift : × So slow × Many parameters - hs=16,hr=16,m=500 - hs=8,hr=8,hw=4,m=500 (170×256) - Time: 0 34 15 - Time: 0 39 54 - classes : 9 - classes : 7 32
  33. 33. Fast (Accelerated) K-mean*  Lemma 1: Let p be a point and let c1 and c2 be centers. c1  If E(c1,c2) ≥ 2E(p,c1) then E(p,c2) ≥ E(p,c1). p  Lemma 2: Let p be a point and let c1 and c2 be centers. Then  E(p,c2) ≥ max{0,E(p,c1) – E(c1,c2)} c2* C.Elkan, ‘Using the triangle inequality to accelerate k-Means.’ In Proceedings of the 20th ICML,Washington DC, 2003. pp 147--153 33
  34. 34. TRICS SystemStructure 1. Segmentation Stage2. Fast K-mean :  with spatial features × structured segmentation  Increase no. clusters k=3 k=9 34
  35. 35. TRICS SystemStructure 1. Segmentation Stage2. Fast K-mean :  without spatial features × scattered regions of same cluster  disjoint region separation 1 1 1 2 2 3 1 4 3 5 before after 35
  36. 36. TRICS SystemStructure 1. Segmentation Stage2. Fast K-mean : × Small regions (noise)  Small regions elimination Original Before After 36
  37. 37. TRICS SystemStructure 1. Segmentation Stage3. Adaptive Fast K-mean :  Clusters number generation  Minimum region size estimation Fully automatic segmentation technique 37
  38. 38. TRICS System Structure 1. Segmentation Stage a. Clusters number generation : • CEC “Combined Estimation Criterion”*: •If the VRC index, for k clusters, is smaller f n than 98 of the VRC index, for k-1 clusters, WCSS = ∑∑ ( xij − mij ) 2 the CEC is not satisfied. i =1 j =1 f n •If the VRC index, for k clusters, is larger than 102 of the VRC index for k-1 clusters, or TSS = ∑∑ ( xij − M i ) 2 if k=1, the CEC is satisfied. i =1 j =1 BCSS = TSS − WCSS •If the VRC index, for clusters, is smaller than 102 but larger than 98 of the VRC index for (n − k ) BCSS clusters, the CEC is satisfied only if TSS for VRC = k-1 clusters is smaller than 70 of TSS for k (k − 1)WCSS clusters.* D.Charalampidis, T.Kasparis, ‘Wavelet-Based Rotational Invariant Roughness Features forTexture Classification and Segmentation’. IEEE Transactions on Image Processing.Vol.11.No.8 38August 2002
  39. 39. TRICS SystemStructure 1. Segmentation Stage Start k=1 Gray Image Fast k-mean k=k+1 Calculate CEC Segmented Image Yes CEC Satisfied? No Stop… Clusters number =k 39
  40. 40. TRICS SystemStructure 1. Segmentation Stageb. Minimum region size estimation : • Split the disjoint regions. • Count all regions size. • Sort regions size and calculate the step between them. • Select the regions size of step more than the largest image dimension. • Consider the minimum region size. 40
  41. 41. TRICS System Structure 1. Segmentation Stage Original gray wavelets Laws A- F- T- E- EM- Image Features Clusters Regions time time time time time 19 12 42 31 59 Wavelets 4 8 sec sec min sec sec size = 170×256 M=500, 10 EM = 324 11 3 min 31 59 Laws 4 7 sec sec 36 sec sec secA-time : adaptive fast k-mean time, F-time: fixed k fast k-mean time, T-time: traditional k-mean , E-time: elimination time , EM-time : Elimination time with estimating minimum size 41region
  42. 42. TRICS SystemStructure Database set1 The training set consists of 32 classes of Brodatz texture database Each image has a size of 256 × 256. Each image was mirrored horizontally and vertically to produce a 512 ×512 image. The image is split into 16 images of size 128 ×128. 256 × 256 512 × 512 16 × 128 × 128 42
  43. 43. TRICS SystemStructure Database set2 The training set consists of 9 classes ‘cloud, sky, sea, sand, tree, grass, stone, water, and wood’ Each class has number of samples from 12 to 25 samples. These samples are taken from real natural images as random 64x64 rectangles. 43
  44. 44. TRICS SystemStructure Database Database record:  Sample  Class (level1,level2)  58 Features (6 Moment statistics, 4 Co-ocurance measures, 3 Tamura, and “ 15 wavelets mean, 15 wavelets variance , 15 wavelets energy” for five levels wavelets decomposition)  Hue 44
  45. 45. TRICS System Structure 2. Classification Stage Feature extraction (Region based)  Rectangular region: 1. Maximum rectangle 2. 64 x 64 rectangle × Arbitrary shape  Padding rectangle** Ying Liu, Xiaofang Zhou, Wei-Ying Ma, ‘Extracting Texture Featuresfrom Arbitrary-shaped Regions for Image Retrieval‘. 2004 IEEEInternational Conference on Multimedia and Expo., Taipei, Jun. 2004 45
  46. 46. TRICS System Structure 2. Classification Stage  Feature extraction (Region based)  Region based features:  GLCM * measures (Energy, Entropy, Inertia, Homogeneity )  Tamura * (Coarseness, Contrast , Directionality’)  Wavelets coefficients for 5 levels  Mean and variance **  Energy **** P.Howarth, S.Ruger,: Evaluation of texture features for content-based image retrieval. In:proceedings of the International Conference on Image and Video Retrieval, Springer-Verlag (2004)326–324** O. Commowick – C. Lenglet – C. Louchet, ‘Wavelet-Based Texture Classification and Retrieval’2003 found in http://www.tsi.enst.fr/tsi/enseignement/ressources/mti/classif-textures/*** Eka Aulia, ‘Hierarchical Indexing For Region Based Image Retrieval’, Master thesis of Science in 46Industrial Engineering, Louisiana State University and Agricultural and Mechanical College, May 2005
  47. 47. TRICS SystemStructure 2. Classification Stage GLCM and Tamura × Scale variant features, not suitable for natural textures 47
  48. 48. TRICS SystemStructure 2. Classification Stage Wavelets mean and variance : × Values were very scattered and the results were not accurate for most cases. Wavelets energies : Classification accuracy of 92% using “leave- one-out” (each (sub) image is classified one by one so that other (sub) images serve as the training data) method 48
  49. 49. TRICS SystemStructure 2. Classification Stagea. Classification technique  KNN classifier with (k=1,k=5,k=10,k=20)  Distance Metric is L2 “Euclidean distance” 1  2 2 E ( I , J ) =  ∑ I (i ) − J (i )   i   k=5 gives accuracy up to 94% using “N-fold “ (the collection of (sub) images is divided into N disjoint sets, of which N-1 serve as training data in turn and the Nth set is used for testing) 49
  50. 50. TRICS SystemStructure 2. Classification Stage 50
  51. 51. TRICS SystemStructure 3. Coloring Stage Color model conversion  HSV/HSB color model Change in Saturation Change in Brightness Change in Hue Hue = 0, Luminance=0.5 Hue = 0, Saturation = 1 Sat=1, Luminance=1  HSI/HLS color model Change in Saturation Change in Luminance Change in Hue Hue = 0, Luminance=0.5 Hue = 0, Saturation = 1 Sat=1, Luminance=1 51
  52. 52. TRICS SystemStructure 3. Coloring Stage HSV & HSL Channels 52
  53. 53. TRICS SystemStructure 3. Coloring Stage Setting Channels values  Brightness The gray image itself  Hue One hue value for each texture  Saturation HSV: 1- brightness HSI: 0.5(1-lightness) 53
  54. 54. TRICS SystemStructure 3. Coloring Stage 54
  55. 55. TRICS SystemStructure 3. Coloring Stage Outlines  Introduction  Automatic coloring in the literature  TRICS ‘Texture Recognition based Image Coloring System’  Results  Conclusion  Future work 55
  56. 56. Results and ConclusionResults Perfect Results Outlines  Introduction  Automatic coloring in the literature  TRICS ‘Texture Recognition based Image Coloring System’  Results  Conclusion  Future work 56
  57. 57. Results and ConclusionResults Perfect Results 57
  58. 58. Results and Conclusion Results PANN Database: 58
  59. 59. Results and Conclusion ResultsMisclassified results2 classification levels: •if the KNN results in 5 classes “grass, sea, water, grass, sea” •The traditional solution is the class of the grass. •The 2 levels classification solution is sea. •(Sea and water), (trees and grass), (sky and clouds) and (wood and stone) are considered as one class in level one. 59
  60. 60. Results and Conclusion ComparisonsHSV/HSBHIS/HLS 60
  61. 61. Results and ConclusionComparisons Local color transfer Outlines  Introduction  Automatic coloring in the literature Global Image Matching  TRICS ‘Texture Recognition based Image Coloring System’  Results  Conclusion  Future work 61
  62. 62. Results and ConclusionConclusion Outlines We proposed a new computer coloring technique that simulates the human vision in this area.  Introduction  Automatic coloring in the The proposed coloring system is contributed for  literature TRICS ‘Texture coloring gray natural scenes. Recognition based Image Coloring System’ The execution time of TRICS is minimized using  Results Conclusion Fast k-mean segmentation technique and the  Future work results are enhanced by splitting the disjoint regions and by eliminating small regions. 62
  63. 63. Results and ConclusionConclusion Clusters number generation algorithm and the minimum region size estimation algorithm increase the professionalism of the system but also increases the time of the execution. And by using both of them TRICS becomes a fully unsupervised intelligent recognition based coloring system. HSV coloring model is very suitable for our system and the coloring results have good natural look. 63
  64. 64. Results and ConclusionConclusion Outlines We consider our proposed system structure as  Introduction an abstract structure for building any more Automatic  coloring in the intelligent coloring systems for any other types of literature images  TRICS ‘Texture Recognition based Image Coloring System’ Our proposed system results perform the other  Results coloring systems.  Conclusion  Future work 64
  65. 65. Future work Gray image Segmentation A Outlines Features extraction Segmentation (Joint, wavelets, laws,…) (Mean Shift, K-Mean,  Introduction FCM,..)  Automatic Segmented image, Clusters coloring in the literature Classification B  TRICS ‘Texture Features extraction Classification (Co-occurrence, Tamura, Wavelets energies) (K-NN classifier,..) Recognition based Image Samples features Coloring Class labels System’ Classes Hues Database  Results  Conclusion Coloring  Future work C Convert image to Set Hue, Saturation, and Convert to HSV channels Brightness RGB 65 Colored image
  66. 66. Future work Intelligent System for Classifying the image Gray image Segmentation A Features extraction Segmentation (Joint, wavelets, laws,…) (Mean Shift, K-Mean, FCM,..) Segmented image, Clusters Classification B Features extraction Classification (Co-occurrence, Tamura, Wavelets energies) (K-NN classifier,..) Samples featuresAdaptive Learning Class labels SOFM Classes Hues Database Coloring C Convert image to Set Hue, Saturation, and Convert to HSV channels Brightness RGB 66 Colored image
  67. 67. Future work Segmentation and classification stages are research areas and any improvement will increase the accuracy of the system. Using different types of features and training set enables the system for coloring images like manmade images, indoors, and people photos . 67
  68. 68. List Of Publications Noura A.Semary, Mohiy M. Hadhoud, W. S. El-Kilani, and Nabil A. Ismail, “Texture Recognition Based Gray Image Coloring”, The 24th National Radio Science Conference (NRSC2007), pp. C22, March 13-15, 2007, Faculty of Engineering, Ain-Shams Univ., Egypt. 68
  69. 69. ‫…‪Thanks‬‬‫و الحمد لله الذي بفضله‬ ‫تتم الصالحات‬ ‫96‬

×