Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

image processing

semantic improved color imaging application

  • Login to see the comments

image processing

  1. 1. SemanticImproved Color Imaging Applications: Its All About Context Under The Guidance Of : Mr. Ritesh Kumar Presented By: AMAL CHACKO
  2. 2. AVast Challenge/Opportunity [Jonathan Good] 2
  3. 3. AVast Challenge/Opportunity “This suggests that there are,at the very least, aquarter of amillion distinct English words,excluding inflections,and words from technical and regional vocabulary[JonnatohantGocod]overed by the OED ...” 2 [Oxford English Dictionary]
  4. 4. AVast Challenge/Opportunity [Jonathan Good] “This suggests that there are,at the very least, a quarter of a million distinct English words,excluding inflections,and words from technical and regional vocabulary not covered by the OED ...” 2 [Oxford English Dictionary] Novel methods and applications to link digital image content with human language.
  5. 5. Presentation Overview method applications statistical framework −15 −5 −10 0 5 sunset layout Method: 3
  6. 6. Presentation Overview method applications statistical framework −15 −5 −10 0 5 sunset layout 1.semantic image enhancement input grass Applications: Method: 3
  7. 7. Presentation Overview method applications statistical framework −15 −5 −10 0 5 sunset layout 1.semantic image enhancement input grass Applications: Method: 2.color naming periwinkle blue 3
  8. 8. Statistical Framework Link image characteristics with keywords.
  9. 9. Image Database [Huiskes et al.,ACMMM,2010]5 • MIR Flickr database,1 Million annotated images. • Selection based on Flickr’s“interestingness” score. • 1 MegaPixel,assume sRGB. gold,oregoncoast,fojrtstevens,astoria,outside, lightroom,sigma,1020mm,nikon,d40, diamondclassphotographer,grass,yellow,blue,sky, clouds,singlecloud,color,saturated,happy,fieldMeredith_Farmer (cc)
  10. 10. Statistical Framework 1M images + keywords 6
  11. 11. Statistical Framework gold gold 6 996’6883312
  12. 12. Statistical Framework gold gold stevewhis (cc) laura.bell (cc) paige_eliz (cc) Arty Smokes (cc) TW Collins (cc) raketentim (cc) golfnride (cc)Dunechaser (cc) Tal Bright (cc) 7 10863752@N00 (cc) 4 6
  13. 13. Statistical Framework TW Collins (cc) raketentim (cc) stevewhis (cc) laura.bell (cc) Dunechaser (cc) golfnride (cc) paige_eliz (cc) Arty Smokes (cc) Tal Bright (cc) 10863752@N00 (cc) gold gold 0% 3% 5% 70% 8% 30% 90% 10% 2% 9% percentage of yellow pixels 8
  14. 14. Statistical Framework 9 sorted list: 0% 2% 3% 5% 8% 9% 10% 30% 70% 90%
  15. 15. Statistical Framework 9 0% 2% 3% 5% 8% 9% 10% 30% 70% 90% 1 5 sorted list: rank index: ranksum: 2 3 4 6 7 8 9 10 T = 4 + 7 + 9 + 10 = 30
  16. 16. Mann-Whitney-Wilcoxon ranksum test [F. Wilcoxon,Individual comparisons by ranking o2 T = 12 µT = nw(nw + nw + 1) nwnw(nw + nw + 1) 2 cardinalities of both sets nw, nw oT 9 methods,Biometrics Bulletin,1(6):80–83,1945] T —µT 30 —22 z = = 4.69 ⇡ 1.71 Statistical Framework 0% 2% 3% 5% 8% 9% 10% 30% 70% 90% 1 5 sorted list: rank index: ranksum: 2 3 4 6 7 8 9 10 T = 4 + 7 + 9 + 10 = 30
  17. 17. Mann-Whitney-Wilcoxon ranksum test [F. Wilcoxon,Individual comparisons by ranking methods,Biometrics Bulletin,1(6):80–83,1945] o2 T = 12 µT = nw(nw + nw + 1) nwnw(nw + nw + 1) 2 cardinalities of both sets nw, nw oT 4.69 T —µT 30 —22 z = = ⇡ 1.71 Statistical Framework 0% 2% 3% 5% 8% 9% 10% 30% 70% 90% 1 2 3 4 5 sorted list: rank index: ranksum: 6 7 8 9 10 T = 4 + 7 + 9 + 10 = 30 significantly more yellow pixels in gold images.z > 0 9
  18. 18. • • CIELAB histogram 15x15x15 bins. values indicate significance of a keyword w.r.t. to a characteristic. Distribution 10 gold
  19. 19. Other Characteristics Spatial lightness layout. light 1 0 −1 −2 −3 −4 −5 −6 −7 12
  20. 20. Other Characteristics Spatial chroma layout. barn 8 6 4 2 0 −2 −4 −6 13
  21. 21. Other Characteristics Spatial Gabor filter layout. fireworks 5 0 −5 −10 14
  22. 22. Summary 15 • Link any characteristic to any keyword. • Fast and highly scalable: millions of images and thousands of keywords. • Base for subsequent imaging applications with semantic awareness.
  23. 23. Semantic Image Enhancement [Lindner et al.,ACM Multimedia2012,long paper]
  24. 24. Which image is better? 17
  25. 25. dark snow Which image is better? 17
  26. 26. Which image is better? sand 18 sunset
  27. 27. Which image is better? sand sunset No decision possible based on pixel values only. 18
  28. 28. Which image is better? sand sunset No decision possible based on pixel values only. Auto-adjust contrast/colors. 18
  29. 29. Which image is better? sand sunset No decision possible based on pixel values only. Manual editing. 18
  30. 30. Which image is better? sand sunset No decision possible based on pixel values only. Automatic Enhancement with Semantics. 18
  31. 31. Today’s Solutions 19 • Modes: Camera:“portrait”,“nature”,“firework”. Printer:“draft”,“presentation”,“text”.
  32. 32. Today’s Solutions 19 • Modes: Camera:“portrait”,“nature”,“firework”. Printer:“draft”,“presentation”,“text”. • Classification + enhancement: skin,sky or other classes. Park et al.06,Ciocca et al.07,Kaufman et al.12.
  33. 33. Today’s Solutions 19 • Modes: Camera:“portrait”,“nature”,“firework”. Printer:“draft”,“presentation”,“text”. • Classification + enhancement: skin,sky or other classes. Park et al.06,Ciocca et al.07,Kaufman et al.12. •Difficult to scale to large vocabularies.
  34. 34. Semantic Image Enhancement Gray scale tone mapping snow 20
  35. 35. Semantic Image Enhancement Gray scale tone mapping snow blue 20 Color enhancement
  36. 36. Semantic Image Enhancement Gray scale tone mapping snow blue Color enhancement macro Change depth-of-field [Zhuo and Sim,2011] 20
  37. 37. Semantic Image Enhancement Gray scale tone mapping snow blue Color enhancement macro Change depth-of-field [Zhuo and Sim,2011] 20
  38. 38. Semantic Enhancement semantic processing input outputimage component semantic component blue characteristics 21
  39. 39. Semantic Enhancement semantic processing input outputimage component semantic component Blue characteristics semantic component 21
  40. 40. Semantic Component 0 50 200 250 6 5 4 3 2 1 0 100 150 pixel value v al u e z red green blue significance values for rose 22
  41. 41. Semantic Component 0 50 200 250 6 5 4 3 2 1 0 100 150 pixel value v al u e z red green blue 0 200 0 50 100 150 200 250 100 input value outputvalue red green blue identit f 0 = ⇢ 1/ (1 + Sz) 1 + S|z| if z Ç 0 if z < 0 S global scale parameter significance values for rose Tone mapping function f 22
  42. 42. Semantic Enhancement semantic processing input outputimage component blue characteristics image component 0 0 50 250 200 150 100 outputvalue red greenbl identity 100 200 input value 23
  43. 43. Image Component rose 24
  44. 44. Image Component rose weight map ⇥ 24 ! = go ⇤zw . col(p) .⇤1 0 go Gaussian blurring kernel (1%of image diagonal) · ⇥ ⇤1 0 normalization operator
  45. 45. Semantic Enhancement semantic processing input outputimage component blue characteristics 0 0 50 250 200 150 100 outputvalue red greenblue identity 100 200 input value 25
  46. 46. Semantic Enhancement blue 0 0 50 250 200 150 100 outputvalue re greenblue identity 100 200 input value Enhance relevant characteristics in relevant regions. input characteristics output 26 Iout = (1 —! ) · Iin + ! · Itmp Itmp
  47. 47. Semantic Enhancement Enhanced Image Blue
  48. 48. sand
  49. 49. sand
  50. 50. snow
  51. 51. snow
  52. 52. strawberry
  53. 53. strawberry
  54. 54. macro
  55. 55. macro
  56. 56. Automatic Color Naming [Lindner et al.,IS&T CIC 2012] & [Lindner et al.,IS&T CGIV 2012]
  57. 57. Introduction Standard psychophysical color naming experiment: green observer 45
  58. 58. Introduction Standard psychophysical color naming experiment: green observer Our approach: statistical framework green 45
  59. 59. 9000+ Color Names 46 • XKCD color survey,psychophysical experiment.
  60. 60. 9000+ Color Names 47 • XKCD color survey,psychophysical experiment. • 950 English color names + color values.
  61. 61. 9000+ Color Names 48 • XKCD color survey,psychophysical experiment. • 950 English color names + color values. • Translate to 9 other languages: Chinese,French,German,Italian,Japanese,Korean, Portuguese,Russian,and Spanish.
  62. 62. 9000+ Color Names 49 • XKCD color survey,psychophysical experiment. • 950 English color names + color values. • Translate to 9 other languages: Chinese,French,German,Italian,Japanese,Korean, Portuguese,Russian,and Spanish. • Example:柔和的粉红色,soft pink, rose tendre,sanftes pink, rosa tenue,ソフトピンク,부드러운녹색,rosa suave,нежно розовый,rosa suave.
  63. 63. DataAcquisition Google Image:soft pink 50
  64. 64. DataAcquisition Google Image:soft pink 51
  65. 65. DataAcquisition Google Image:soft pink • 100 images per color name. • Language and country restrict. • Assume sRGB encoding. • Almost 1M images. 51
  66. 66. • CIELAB histogram 15x15x15 bins. 52 Distribution soft pink, English
  67. 67. • CIELAB histogram 15x15x15 bins. Distribution soft pink, English sRGB:238,197,203 52
  68. 68. Soft Pink 柔和的粉红色,cn soft pink,en rose tendre,fr sanftes pink,de rosatenue,it ソフトピンク,jp 부드러운녹색,ko rosasuave,pt нежно розовый,ru rosasuave,es 53
  69. 69. Soft Pink 柔和的粉红色,cn soft pink,en rose tendre,fr sanftes pink,de rosatenue,it ソフトピンク,jp 부드러운녹색,ko rosasuave,pt rosasuave,es 54
  70. 70. Soft Pink 柔和的粉红色,cn soft pink,en rose tendre,fr sanftes pink,de rosatenue,it ソフトピンク,jp 부드러운녹색,ko rosasuave,pt rosasuave,es Language and country restrict. 54
  71. 71. Color Estimations Chinese English French German Italian Japanese Korean Portuguese Russian Spanish 55
  72. 72. Conclusions & FutureWork sunset layout 5 0 −5 −10 −15 61 Easily scalable statistical framework.
  73. 73. Conclusions & FutureWork sunset layout 5 0 −5 −10 −15 Easily scalable statistical framework. input grass 61 Semantic image enhancement for tone-mapping,color and depth-of-field.
  74. 74. Conclusions & FutureWork sunset layout 5 0 −5 −10 −15 Easily scalable statistical framework. input grass Semantic image enhancement for tone-mapping,color and depth-of-field. periwinkle blue Automatic color naming and an interactive online color thesaurus. 61
  75. 75. •Du-Sik Park,Youngshin Kwak,Hyunwook Ok and Chang-Yeong Kim,Preferred skin color reproduction on the display,JEI,2006. •Gianluigi Ciocca,Claudio Cusano,Francesca Gasparini and Raimondo Schettini,ContentAware Image Enhancement,Artificial Intelligence and Human-Oriented Computing,2007. •Liad Kaufman.Dani Lischinski and MichaelWerman,Content-AwareAutomatic Photo Enhancement,Computer Graphics Forum,2012. •BaoyuanWang,YizhouYu,Tien-TsinWong,Chun Chen andYing-Qing Xu, Data-Driven Image ColorTheme Enhancement,ACM SIGGRAPH,2010. •NailaMurray,Sandra Skaff and Luca Marchesotti,Towards Automatic ConceptTransfer, SIGGRAPH/Eurographics Symposium on Non-PhotorealisticAnimation and Rendering,2011. •FrankWilcoxon,Individual Comparisons by Ranking Methods,Biometrics Bulletin,1945. •Shaojie Zhuo andTerence Sim,Defocus map estimation from a single image,Pattern Recognition,2011. •Sung Ju Hwang,Ashish Kapoor and Sing Bing Kang,Context-BasedAutomatic Local Image Enhancement,ECCV,2012. Reference

×