<ul><li>Image Fusion for  </li></ul><ul><li>Context Enhancement </li></ul><ul><li>and Video Surrealism </li></ul>Adrian Il...
 
Dark Bldgs Reflections on bldgs Unknown shapes
‘ Well-lit’ Bldgs Reflections in bldgs windows Tree, Street shapes
Background is captured from day-time scene using the same fixed camera  Night Image  Day Image Context Enhanced Image
Mask is automatically computed from scene contrast
But, Simple Pixel Blending Creates  Ugly Artifacts
Pixel Blending
Pixel Blending Our Method : Integration of  blended Gradients
Outline <ul><li>Context Enhancement </li></ul><ul><li>Gradient-based Fusion </li></ul><ul><li>Video Enhancement </li></ul>...
Nighttime image Daytime image Gradient field Importance image W Final result Gradient field Mixed gradient field G 1 G 1 G...
Reconstruction from Gradient Field <ul><li>Problem: minimize error   I’ – G| </li></ul><ul><li>Estimate I’ so that </l...
Why Gradient-based Approach <ul><li>Comparison of intensity values are important </li></ul><ul><li>Maintain gradients to c...
Comparison <ul><li>Average </li></ul><ul><ul><li>Subtle details are lost </li></ul></ul><ul><li>Pixel-wise blending </li><...
Issues <ul><li>Boundary conditions </li></ul><ul><li>Color shifts </li></ul>
Boundary Conditions <ul><li>Assumed Neumann condition at borders,  </li></ul><ul><li>  I’  ·  N ...
Color Shift <ul><li>Mixing dissimilar images </li></ul><ul><li>Goal: final image appearance matches input images at corres...
 
Outline <ul><li>Context Enhancement </li></ul><ul><li>Gradient-based Fusion </li></ul><ul><li>Video Enhancement </li></ul>...
 
 
Overview of Process Enhanced video  Note: exit ramp, lane dividers, buildings not visible in original night video, but cle...
Algorithm Frame  N Daytime image TimeAveraged importance mask Gradient field Final result Processed binary mask Gradient f...
Outline <ul><li>Context Enhancement </li></ul><ul><li>Gradient-based Fusion </li></ul><ul><li>Video Enhancement </li></ul>...
Related Work <ul><li>Spatio-temporal Composition </li></ul><ul><ul><li>Duchamp ( Nude descending a staircase ) </li></ul><...
Rene Magritte,  ‘Empire of the Light’ Surrealism
Outline <ul><li>Context Enhancement </li></ul><ul><li>Gradient-based Fusion </li></ul><ul><li>Video Enhancement </li></ul>...
Time-lapse Mosaics Maggrite Stripes time
Time Lapse Mosaic
Time Lapse Mosaic
t
Sunrise at Night
BiSolar System
Discussion <ul><li>User Experience </li></ul><ul><ul><li>More effective in conveying scene context </li></ul></ul><ul><ul>...
Image Fusion for Context Enhancement <ul><li>Nonrealistic but comprehensible context </li></ul><ul><li>Fusion using multip...
Upcoming SlideShare
Loading in …5
×

Raskar Npar04final

408 views

Published on

Published in: Technology, Business
0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total views
408
On SlideShare
0
From Embeds
0
Number of Embeds
7
Actions
Shares
0
Downloads
14
Comments
0
Likes
0
Embeds 0
No embeds

No notes for slide
  • &lt;Title&gt; Hello. My name is Mike Brown, and I am here to present a paper by Adrian Ilie, Ramesh Raskar and Jingyi Yu on Gradient-Domain Context Enhancement for Fixed Cameras. I will try to answer your questions, but I suggest that you address the more specific ones to the authors themselves, by email.
  • Raskar Npar04final

    1. 1. <ul><li>Image Fusion for </li></ul><ul><li>Context Enhancement </li></ul><ul><li>and Video Surrealism </li></ul>Adrian Ilie UNC Chapel Hill Ramesh Raskar Mitsubishi Electric Research Labs, (MERL) Jingyi Yu MIT
    2. 3. Dark Bldgs Reflections on bldgs Unknown shapes
    3. 4. ‘ Well-lit’ Bldgs Reflections in bldgs windows Tree, Street shapes
    4. 5. Background is captured from day-time scene using the same fixed camera Night Image Day Image Context Enhanced Image
    5. 6. Mask is automatically computed from scene contrast
    6. 7. But, Simple Pixel Blending Creates Ugly Artifacts
    7. 8. Pixel Blending
    8. 9. Pixel Blending Our Method : Integration of blended Gradients
    9. 10. Outline <ul><li>Context Enhancement </li></ul><ul><li>Gradient-based Fusion </li></ul><ul><li>Video Enhancement </li></ul><ul><li>Surrealism </li></ul>
    10. 11. Nighttime image Daytime image Gradient field Importance image W Final result Gradient field Mixed gradient field G 1 G 1 G 2 G 2 x Y x Y I 1 I 2 G G x Y
    11. 12. Reconstruction from Gradient Field <ul><li>Problem: minimize error   I’ – G| </li></ul><ul><li>Estimate I’ so that </li></ul><ul><li> </li></ul><ul><li>G =   I’ </li></ul><ul><li>Poisson equation </li></ul><ul><li>  I’ = div G </li></ul><ul><li>Full multigrid </li></ul><ul><li>solver </li></ul>I’ G X G Y
    12. 13. Why Gradient-based Approach <ul><li>Comparison of intensity values are important </li></ul><ul><li>Maintain gradients to capture local variations </li></ul><ul><li>Directly solve for desired gradients </li></ul><ul><li>Maintain subtle details </li></ul><ul><li>Mix dissimilar images </li></ul><ul><li>No need for precise segmentation </li></ul>
    13. 14. Comparison <ul><li>Average </li></ul><ul><ul><li>Subtle details are lost </li></ul></ul><ul><li>Pixel-wise blending </li></ul><ul><ul><li>Sharp transitions </li></ul></ul>
    14. 15. Issues <ul><li>Boundary conditions </li></ul><ul><li>Color shifts </li></ul>
    15. 16. Boundary Conditions <ul><li>Assumed Neumann condition at borders, </li></ul><ul><li>  I’ · N = 0 , </li></ul><ul><li>Enforced by haloing image with blacks </li></ul>
    16. 17. Color Shift <ul><li>Mixing dissimilar images </li></ul><ul><li>Goal: final image appearance matches input images at corresponding pixels </li></ul><ul><li>I final (x,y) = c 1  I poisson (x,y) + c 2 </li></ul><ul><li>Solve </li></ul><ul><ul><li> W i (x,y) I original (x,y) = c 1  I poisson (x,y) + c 2 </li></ul></ul><ul><li>Each color channel reconstructed separately </li></ul>
    17. 19. Outline <ul><li>Context Enhancement </li></ul><ul><li>Gradient-based Fusion </li></ul><ul><li>Video Enhancement </li></ul><ul><li>Surrealism </li></ul>
    18. 22. Overview of Process Enhanced video Note: exit ramp, lane dividers, buildings not visible in original night video, but clearly seen here. Day time image: By averaging 5 seconds of day video Original night time traffic camera 320x240 video Mask frame (for frame above): Encodes pixel with intensity change Input Output
    19. 23. Algorithm Frame N Daytime image TimeAveraged importance mask Gradient field Final result Processed binary mask Gradient field Frame N-1 Mixed gradient field
    20. 24. Outline <ul><li>Context Enhancement </li></ul><ul><li>Gradient-based Fusion </li></ul><ul><li>Video Enhancement </li></ul><ul><li>Related Work </li></ul><ul><li>Surrealism </li></ul>
    21. 25. Related Work <ul><li>Spatio-temporal Composition </li></ul><ul><ul><li>Duchamp ( Nude descending a staircase ) </li></ul></ul><ul><ul><li>Freeman 2002 </li></ul></ul><ul><ul><li>Fels 1999, Klein 2002, Cohen 2003 </li></ul></ul><ul><li>Gradient-based Techniques </li></ul><ul><ul><li>Multi-spectral: Socolinsky 1999 </li></ul></ul><ul><ul><li>Shadow removal: Weiss 2001 </li></ul></ul><ul><ul><li>High dynamic range: Fattal 2002 </li></ul></ul><ul><ul><li>Image editing: Perez 2003 </li></ul></ul><ul><ul><li>Some at Siggraph’04 </li></ul></ul>
    22. 26. Rene Magritte, ‘Empire of the Light’ Surrealism
    23. 27. Outline <ul><li>Context Enhancement </li></ul><ul><li>Gradient-based Fusion </li></ul><ul><li>Video Enhancement </li></ul><ul><li>Surrealism </li></ul>
    24. 28. Time-lapse Mosaics Maggrite Stripes time
    25. 29. Time Lapse Mosaic
    26. 30. Time Lapse Mosaic
    27. 31. t
    28. 32. Sunrise at Night
    29. 33. BiSolar System
    30. 34. Discussion <ul><li>User Experience </li></ul><ul><ul><li>More effective in conveying scene context </li></ul></ul><ul><ul><li>‘ Dreamy’ appearance </li></ul></ul><ul><ul><li>Nonrealistic : False conditions </li></ul></ul><ul><li>Applications </li></ul><ul><ul><li>Tools for artists </li></ul></ul><ul><ul><li>Surveillance </li></ul></ul><ul><ul><li>Amusement park rides </li></ul></ul><ul><li>Performance </li></ul><ul><ul><li>~1 sec/frame for 320x240 </li></ul></ul><ul><ul><li>~ 3 min for 4Mpixel image </li></ul></ul>
    31. 35. Image Fusion for Context Enhancement <ul><li>Nonrealistic but comprehensible context </li></ul><ul><li>Fusion using multiple images </li></ul><ul><ul><li>Enhancing night images with day bgrnd </li></ul></ul><ul><li>Gradient-based fusion </li></ul><ul><li>Video surrealism tools </li></ul>t

    ×