Color Harmonization For Videos

737 views
646 views

Published on

0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total views
737
On SlideShare
0
From Embeds
0
Number of Embeds
7
Actions
Shares
0
Downloads
7
Comments
0
Likes
0
Embeds 0
No embeds

No notes for slide
  • Not all colors match with every other color Such image can be chaotic or boring Our notion of colors very much depends on place, time, context and many other factors Lack of color harmony may leave disturbing impact on the observer Hence here our ultimate goal is to improve the visual harmony of the image.
  • Why do we need color harmonization for videos? As we all know videos are nothing but set of images, So the problem is compounded as we can have Rapidly changing scenes Indoor and outdoor scenes Changing illumination Back to back un harmonized scenes So our goal is to generate a visually enhance video which will create a pleasant feel on the observer.
  • We start with images Our approach We make use of template to harmonize images. If entire hue of an image is residing with the shaded area of a particular template then we say that the image is harmonized under that particular template (explain the templates)
  • (explain potential function) (theta ot) (theta oH) (color adjustment)
  • Segmentation problem This is caused due to splitting of region with different shades of a color into regions with extreme colors after color adjustment As shown in the image.
  • Our solution simple and faster. We consider the hue histogram of the given image. The segmentation is possibly bcoz of point of splitting is passing through _________________ We would like this splitting point to be present point on the histogram which is relatively empty. Our algorithm chooses such a point.. Our algorithm is slightly less accurate but faster than graph cut base algorithms previously used
  • we extend the method for color harmonization to videos. Videos are nothing but set of frames We harmonize each frames individually irrespective of other frames
  • We observed flickering artifacts when the frames are stitched back. This is mainly due to change in theta and t at subsequent frames. Due to this change colors are shifted indefinitely it creates a disturbing effect on the observer. Even though all the frames are harmonized individually they don’t create pleasant effect Hence we need to correlate the neighboring frames so that flickering artifacts are removed, Which is achieved by
  • grouping
  • We take the mean of the hue histogram of the frames falling the same group This mean is further used to calculate the theta This newly calculated theta is used for harmonizing the images under that particular group. Hence within image we hardly see any flickering In this we slightly compromise upon the harmonization of individual frame but over effect of the given frame is pleasant But still we observe flickering at the end and beginning of the group..
  • Color Harmonization For Videos

    1. 1. Color Harmonization for Videos Nikhil Sawant and Niloy J. Mitra Dept. of CSE, Indian Institute of Technology, Delhi
    2. 2. Color Harmonization ? Color Harmonization for Videos Nilkhil Sawant and Niloy J. Mitra
    3. 3. <ul><li>Not all colors match </li></ul><ul><li>Colors can be ‘chaotic’ or ‘boring’ </li></ul><ul><li>Overall impact can be disturbing </li></ul><ul><li>Notion of harmony depends on place, time, context, etc. </li></ul><ul><li>Ultimate goal </li></ul><ul><li>Enhance visual harmony </li></ul>Color Harmonization ? Color Harmonization for Videos Nilkhil Sawant and Niloy J. Mitra
    4. 4. <ul><li>Videos are sets of images </li></ul><ul><li>Problematic due to: </li></ul><ul><ul><li>Rapidly changing scenes </li></ul></ul><ul><ul><li>Indoor and outdoor scenes </li></ul></ul><ul><ul><li>Changing illumination </li></ul></ul><ul><ul><li>Back to back un-harmonized scene </li></ul></ul><ul><li>Ultimate goal </li></ul><ul><li>Enhance visual harmony of videos </li></ul>Color Harmonization for Videos ? Color Harmonization for Videos Nilkhil Sawant and Niloy J. Mitra
    5. 5. Images [Cohen-Or et al., Color Harmonization, SIGGRAPH, 2006] Color Harmonization for Videos Nilkhil Sawant and Niloy J. Mitra
    6. 6. <ul><li>Templates for harmonization </li></ul><ul><li>Need for optimization </li></ul>Our approach Color Harmonization for Videos Nilkhil Sawant and Niloy J. Mitra [Johannes Itten. The Art of Color: the subjective experience and objective rationale of color. Van Nostrand Reinhold, 1973] Template I Template V Template L Template I Template T Template Y Template X Template N
    7. 7. <ul><li>Templates for harmonization </li></ul><ul><li>Potential function: </li></ul><ul><ul><li>P( θ t , I) = [# image pixels in the shaded region] </li></ul></ul><ul><li>θ ot = argmax(P( θ t , I)) θ є [0 – 360 ° ] </li></ul><ul><li>θ H = argmax(P( θ ot , I)) T є {i, V, L, I, T, Y, X} </li></ul><ul><li>Linear Color adjustment </li></ul>Our approach Color Harmonization for Videos Nilkhil Sawant and Niloy J. Mitra θ t θ o t
    8. 8. Result Color Harmonization for Videos Nilkhil Sawant and Niloy J. Mitra Template X
    9. 9. Result Color Harmonization for Videos Nilkhil Sawant and Niloy J. Mitra Template V
    10. 10. <ul><li>Splitting of color within same region </li></ul>Segmentation problem Color Harmonization for Videos Nilkhil Sawant and Niloy J. Mitra Template V
    11. 11. <ul><li>Splitting of color within same region </li></ul><ul><li>Solution - Find best position for splitting </li></ul>Segmentation problem Color Harmonization for Videos Nilkhil Sawant and Niloy J. Mitra
    12. 12. Segmentation problem Color Harmonization for Videos Nilkhil Sawant and Niloy J. Mitra
    13. 13. <ul><li>Less accurate but faster than traditional approaches </li></ul>Segmentation problem Nilkhil Sawant and Niloy J. Mitra Color Harmonization for Videos
    14. 14. Videos Color Harmonization for Videos Nilkhil Sawant and Niloy J. Mitra
    15. 15. CH extended for videos <ul><li>Videos are sets of images (frames) </li></ul><ul><li>Naïve approach: each frame independently harmonized </li></ul>Color Harmonization for Videos Nilkhil Sawant and Niloy J. Mitra
    16. 16. CH extended for videos <ul><li>Videos are sets of images (frames) </li></ul><ul><li>Naïve approach: each frame independently harmonized </li></ul><ul><li>Flickering artifacts observed </li></ul><ul><li>Noticeable change in color due to large variation in θ </li></ul><ul><li>Need to correlate the adjacent frames </li></ul><ul><li>Grouping of frames </li></ul>Color Harmonization for Videos Nilkhil Sawant and Niloy J. Mitra
    17. 17. Grouping Color Harmonization for Videos Nilkhil Sawant and Niloy J. Mitra
    18. 18. Grouping <ul><li>Mean color for group of frames </li></ul><ul><li>Calculation of θ g considering mean color </li></ul><ul><li>Frames under a group harmonized by θ g </li></ul><ul><li>No flickering within a group </li></ul><ul><li>Flickering at the group junction </li></ul>Color Harmonization for Videos Nilkhil Sawant and Niloy J. Mitra
    19. 19. Overlapping Color Harmonization for Videos Nilkhil Sawant and Niloy J. Mitra
    20. 20. Overlapping <ul><li>Adjacent groups contain common frames </li></ul><ul><li>Similar approach to grouping </li></ul><ul><li>Common frames harmonized according mean θ g of two groups (smoother transition) </li></ul><ul><li>Flickering artifacts between group are reduced </li></ul><ul><li>Change of scene still cause flickering artifacts </li></ul>Color Harmonization for Videos Nilkhil Sawant and Niloy J. Mitra
    21. 21. Change of scene <ul><li>Change of camera </li></ul><ul><li>Change of camera view port </li></ul><ul><li>Change in illumination </li></ul><ul><li>Larger change in colors </li></ul><ul><li>We categorize all such frames as I-frames </li></ul>Color Harmonization for Videos Nilkhil Sawant and Niloy J. Mitra
    22. 22. Change of scene detection(I-frames) <ul><li>Calculating MSE for each frame </li></ul><ul><li>MSE is measure of color change </li></ul><ul><li>Threshold is selected </li></ul>Color Harmonization for Videos Nilkhil Sawant and Niloy J. Mitra
    23. 23. I-frames + overlapping <ul><li>An I-frame causes start of new group </li></ul><ul><li>Between two I-frames overlapping method used </li></ul>Color Harmonization for Videos Nilkhil Sawant and Niloy J. Mitra I-frames + overlapping Naïve approach
    24. 24. Results Nilkhil Sawant and Niloy J. Mitra Color Harmonization for Videos Original Video Harmonized Video
    25. 25. Two parameter variation <ul><li>Slight variation in the template is allowed </li></ul><ul><li>Allow template to adjust according to image hue histogram </li></ul><ul><li>Harmonization with minimal color change </li></ul>Color Harmonization for Videos Nilkhil Sawant and Niloy J. Mitra
    26. 26. Two parameter variation Template X Color Harmonization for Videos Nilkhil Sawant and Niloy J. Mitra
    27. 27. Two parameter variation <ul><li>Large deviation in template may affect the harmonization </li></ul>Color Harmonization for Videos Nilkhil Sawant and Niloy J. Mitra
    28. 28. Conclusion <ul><li>Faster technique for color harmonization of images </li></ul><ul><li>Less reliable but faster technique for segmentation problem </li></ul><ul><li>Introduction of harmonization technique to Videos </li></ul><ul><li>Reduction in flickering artifacts </li></ul><ul><li>Introduction of Two parameter variation method </li></ul>Color Harmonization for Videos Nilkhil Sawant and Niloy J. Mitra
    29. 29. Thank you Original Video Harmonized Video Results Nilkhil Sawant and Niloy J. Mitra Color Harmonization for Videos
    30. 30. Thank you Color Harmonization for Videos Nilkhil Sawant and Niloy J. Mitra

    ×