Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Introduction to Camera Challenges - Ramesh Raskar

1,580 views

Published on

Published in: Art & Photos, Technology
  • DOWNLOAD FULL BOOKS INTO AVAILABLE FORMAT ......................................................................................................................... ......................................................................................................................... 1.DOWNLOAD FULL PDF EBOOK here { https://tinyurl.com/y8nn3gmc } ......................................................................................................................... 1.DOWNLOAD FULL EPUB Ebook here { https://tinyurl.com/y8nn3gmc } ......................................................................................................................... 1.DOWNLOAD FULL doc Ebook here { https://tinyurl.com/y8nn3gmc } ......................................................................................................................... 1.DOWNLOAD FULL PDF EBOOK here { https://tinyurl.com/y8nn3gmc } ......................................................................................................................... 1.DOWNLOAD FULL EPUB Ebook here { https://tinyurl.com/y8nn3gmc } ......................................................................................................................... 1.DOWNLOAD FULL doc Ebook here { https://tinyurl.com/y8nn3gmc } ......................................................................................................................... ......................................................................................................................... ......................................................................................................................... .............. Browse by Genre Available eBooks ......................................................................................................................... Art, Biography, Business, Chick Lit, Children's, Christian, Classics, Comics, Contemporary, Cookbooks, Crime, Ebooks, Fantasy, Fiction, Graphic Novels, Historical Fiction, History, Horror, Humor And Comedy, Manga, Memoir, Music, Mystery, Non Fiction, Paranormal, Philosophy, Poetry, Psychology, Religion, Romance, Science, Science Fiction, Self Help, Suspense, Spirituality, Sports, Thriller, Travel, Young Adult,
       Reply 
    Are you sure you want to  Yes  No
    Your message goes here

Introduction to Camera Challenges - Ramesh Raskar

  1. 1. © 2004 Marc Levoy The CityBlock Project Precursor to Google Streetview Maps
  2. 2. Image Fusion & ReconstructionImage Fusion & Reconstruction • Single photo:Single photo: forces narrow tradeoffs:forces narrow tradeoffs: – Focus, Exposure, aperture, time, sensitivity, noise,Focus, Exposure, aperture, time, sensitivity, noise, – Usual result: Incomplete visual appearance.Usual result: Incomplete visual appearance. Multiple photosMultiple photos, assorted settings, assorted settings for Optics, Sensor, Lighting, Processingfor Optics, Sensor, Lighting, Processing • Fusion:Fusion: ‘Merge the best parts’‘Merge the best parts’ • Reconstruction:Reconstruction: Detect photo changes;Detect photo changes; compute scene invariantscompute scene invariants
  3. 3. High Dynamic Range ImagingHigh Dynamic Range Imaging • Cameras have limited dynamic range Small Exposure image, dark inside 1/500 sec Large exposure image, saturated outside ¼ sec Images from Raanan Fattal
  4. 4. High Dynamic Range ImagingHigh Dynamic Range Imaging • Combine images at different exposures • Exposure Bracketing • [Mann and Picard 95, Debevec et al 96] Images from Raanan Fattal
  5. 5. How could we put all this information into one image ?
  6. 6. Tone Map 20 bit image for 8 bit DisplayTone Map 20 bit image for 8 bit Display
  7. 7. input smoothed (structure, large scale) residual (texture, small scale) Gaussian Convolution BLUR HALOS Naïve Approach: Gaussian Blur
  8. 8. Impact of Blur and Halos • If the decomposition introduces blur and halos, the final result is corrupted. Sample manipulation: increasing texture (residual × 3)
  9. 9. input smoothed (structure, large scale) residual (texture, small scale) edge-preserving: Bilateral Filter Bilateral Filter: no Blur, no Halos
  10. 10. input
  11. 11. increasing texture with Gaussian convolution H A L O S
  12. 12. increasing texture with bilateral filter N O H A L O S
  13. 13. Bilateral Filter on 1D Signal BF
  14. 14. p Our Strategy Reformulate the bilateral filter – More complex space:  Homogeneous intensity  Higher-dimensional space – Simpler expression: mainly a convolution  Leads to a fast algorithm weights applied to pixels
  15. 15. Attenuate High GradientsAttenuate High Gradients I(x) 1 105 1 Intensity I(x) 1 105 Intensity Maintain local detail at the cost of global range Fattal et al Siggraph 2002
  16. 16. Attenuate High GradientsAttenuate High Gradients I(x) 1 105 G(x) 1 105 Intensity Gradient I(x) 1 105 Intensity Maintain local detail at the cost of global range Fattal et al Siggraph 2002
  17. 17. Attenuate High GradientsAttenuate High Gradients I(x) 1 105 G(x) 1 105 Intensity Gradient I(x) 1 105 Intensity Keep low gradients Fattal et al Siggraph 2002
  18. 18. Gradient Compression in 1DGradient Compression in 1D
  19. 19. Gradient Domain CompressionGradient Domain Compression HDR Image L Log L Gradient Attenuation Function G Multiply 2D Integration Gradients Lx,Ly
  20. 20. Grad X Grad Y New Grad X New Grad Y 2D Integration Intensity Gradient ManipulationIntensity Gradient Manipulation Gradient Processing A Common Pipeline This Section Next Section
  21. 21. Grad X Grad Y New Grad X New Grad Y 2D Integration Gradient Processing
  22. 22. Local Illumination ChangeLocal Illumination Change Original gradient field: Original Image: f * f∇ Modified gradient field: v Perez et al. Poisson Image editing, SIGGRAPH 2003
  23. 23. Ambient Flash Self-Reflections and Flash HotspotSelf-Reflections and Flash Hotspot Hands Face Tripod
  24. 24. ResultAmbient Flash Reflection LayerReflection Layer Hands Face Tripod
  25. 25. Intensity Gradient VectorIntensity Gradient Vector ProjectionProjection [Agrawal, Raskar, Nayar, Li SIGGRAPH 2005][Agrawal, Raskar, Nayar, Li SIGGRAPH 2005]
  26. 26. Intensity Gradient Vectors in Flash and Ambient ImagesIntensity Gradient Vectors in Flash and Ambient Images Same gradient vector direction Flash Gradient Vector Ambient Gradient Vector Ambient Flash No reflections
  27. 27. Reflection Ambient Gradient Vector Different gradient vector direction With reflections Ambient Flash Flash Gradient Vector
  28. 28. Residual Gradient Vector Intensity Gradient Vector Projection Result Gradient Vector Result Residual Reflection Ambient Gradient Vector Flash Gradient Vector Ambient Flash
  29. 29. Flash Projection = Result Residual = Reflection Layer Co-located Artifacts Ambient
  30. 30. Recovering foreground layerRecovering foreground layer – Find tensor based on background image – Transform gradient field of foreground image Foreground maskImage Difference
  31. 31. Dark Bldgs Reflections on bldgs Unknown shapes
  32. 32. ‘Well-lit’ Bldgs Reflections in bldgs windows Tree, Street shapes
  33. 33. Background is captured from day-time scene using the same fixed camera Night Image Day Image Context Enhanced Image
  34. 34. Mask is automatically computed from scene contrast
  35. 35. But, Simple Pixel Blending Creates Ugly Artifacts
  36. 36. Pixel Blending
  37. 37. Pixel Blending Our Method: Integration of blended Gradients
  38. 38. Nighttime imageNighttime image Daytime imageDaytime image Gradient fieldGradient field ImportanceImportance image Wimage W FinalresultFinalresult Gradient fieldGradient field Mixed gradient fieldMixed gradient field GG11 GG11 GG22 GG22 xx YY xx YY II11 I2 GG GG xx YY
  39. 39. Reconstruction from Gradient FieldReconstruction from Gradient Field • Problem: minimize error |∇ I’ – G| • Estimate I’ so that G = ∇ I’ • Poisson equation ∇ 2 I’ = div G • Full multigrid solver I’I’ GGXX GGYY
  40. 40. Rene Magritte, ‘Empire of the Light’ Surrealism
  41. 41. actual photomontageset of originals perceived
  42. 42. Source images Brush strokes Computed labeling Composite
  43. 43. Brush strokes Computed labeling
  44. 44. • No Flash:No Flash: Candle warmth, but high noiseCandle warmth, but high noise • Flash:Flash: low noise, but no candle warmthlow noise, but no candle warmth Photography: Full of Tradeoffs...Photography: Full of Tradeoffs... No-flash Flash
  45. 45. Image A: Warm, shadows, but too Noisy (too dim for a good quick photo) No-flash
  46. 46. Image B: Cold, Shadow-free, Clean (flash: simple light, ALMOST no shadows)
  47. 47. MERGE BEST OF BOTH: apply ‘Cross Bilateral’ or ‘Joint Bilateral’
  48. 48. (it really is much better!)
  49. 49. Image Fusion & ReconstructionImage Fusion & Reconstruction • Single photo:Single photo: forces narrow tradeoffs:forces narrow tradeoffs: – Focus, Exposure, aperture, time, sensitivity, noise,Focus, Exposure, aperture, time, sensitivity, noise, – Usual result: Incomplete visual appearance.Usual result: Incomplete visual appearance. Multiple photosMultiple photos, assorted settings, assorted settings for Optics, Sensor, Lighting, Processingfor Optics, Sensor, Lighting, Processing • Fusion:Fusion: ‘Merge the best parts’‘Merge the best parts’ • Reconstruction:Reconstruction: Detect photo changes;Detect photo changes; compute scene invariantscompute scene invariants
  50. 50. The Media Lab Camera Culture Epsilon Photography Capture multiple photos, each with slightly different camera parameters. • Exposure settings • Spectrum/color settings • Focus settings • Camera position • Scene illumination
  51. 51. FUSION: Best-Focus DistanceFUSION: Best-Focus Distance Agrawala et al., Digital Photomontage SIGGRAPH 2004 NEARNEAR
  52. 52. FUSION: Best-Focus DistanceFUSION: Best-Focus Distance Agrawala et al., Digital Photomontage SIGGRAPH 2004 FARFAR
  53. 53. FUSION: Best-Focus DistanceFUSION: Best-Focus Distance Agrawala et al., Digital Photomontage SIGGRAPH 2004
  54. 54. FUSION: Best-Focus DistanceFUSION: Best-Focus Distance Agrawala et al., Digital Photomontage SIGGRAPH 2004
  55. 55. FUSION: Best-Focus DistanceFUSION: Best-Focus Distance Agrawala et al., Digital Photomontage SIGGRAPH 2004
  56. 56. FUSION: Best-Focus DistanceFUSION: Best-Focus Distance Agrawala et al., Digital Photomontage SIGGRAPH 2004
  57. 57. FUSION: Best-Focus DistanceFUSION: Best-Focus Distance Agrawala et al., Digital Photomontage SIGGRAPH 2004
  58. 58. FUSION: Best-Focus DistanceFUSION: Best-Focus Distance Agrawala et al., Digital Photomontage SIGGRAPH 2004
  59. 59. FUSION: Best-Focus DistanceFUSION: Best-Focus Distance Agrawala et al., Digital Photomontage SIGGRAPH 2004
  60. 60. FUSION: Best-Focus DistanceFUSION: Best-Focus Distance Agrawala et al., Digital Photomontage SIGGRAPH 2004
  61. 61. FUSION: Best-Focus DistanceFUSION: Best-Focus Distance Agrawala et al., Digital Photomontage SIGGRAPH 2004
  62. 62. FUSION: Best-Focus DistanceFUSION: Best-Focus Distance Agrawala et al., Digital Photomontage SIGGRAPH 2004
  63. 63. FUSION: Best-Focus DistanceFUSION: Best-Focus Distance Agrawala et al., Digital Photomontage SIGGRAPH 2004
  64. 64. FUSION: Best-Focus DistanceFUSION: Best-Focus Distance Agrawala et al., Digital Photomontage SIGGRAPH 2004
  65. 65. FUSION: Best-Focus DistanceFUSION: Best-Focus Distance Agrawala et al., Digital Photomontage SIGGRAPH 2004
  66. 66. Source images ‘Graph Cuts’ Solution FUSION Agrawala et al., Digital Photomontage SIGGRAPH 2004
  67. 67. What else can we extend?What else can we extend? Film-Like Camera Parameters:Film-Like Camera Parameters: • Field of View: image stitching for panoramasField of View: image stitching for panoramas • Dynamic Range:Dynamic Range: Radiance MapsRadiance Maps • Frame Rate: Interleaved VideoFrame Rate: Interleaved Video • Resolution: ‘Super-resolution’ methodsResolution: ‘Super-resolution’ methods Visual Appearance & Content:Visual Appearance & Content: • Tone Map:Tone Map: Detail in every shadow and highlightDetail in every shadow and highlight • Color2grey:Color2grey: KeepKeep allall color changes in grayscalecolor changes in grayscale • Temporal Continuity: Space-time fusionTemporal Continuity: Space-time fusion • Viewpoint Constraints:Viewpoint Constraints: Multiple COP imagesMultiple COP images and more…and more…
  68. 68. The Media Lab Camera Culture Epsilon Photography Capture multiple photos, each with slightly different camera parameters. • Exposure settings • Spectrum/color settings • Focus settings • Camera position • Scene illumination
  69. 69. The Media Lab Camera Culture Project Ideas
  70. 70. © 2004 Marc Levoy The CityBlock Project Precursor to Google Streetview Maps
  71. 71. What is ‘interesting’ here? Social voting in the real world = ‘popular’
  72. 72. Vein ViewerVein Viewer (Luminetx)(Luminetx) Near-IR camera locates subcutaneous veins and projectNear-IR camera locates subcutaneous veins and project their location onto the surface of the skin.their location onto the surface of the skin. Coaxial IR cameraCoaxial IR camera + Projector+ Projector Coaxial IR cameraCoaxial IR camera + Projector+ Projector
  73. 73. Focus Adjustment: Sum of Bundles
  74. 74. http://www.mne.psu.edu/psgdl/FSSPhotoalbum/index1.htm
  75. 75. Varying PolarizationVarying Polarization Yoav Y. Schechner, Nir Karpel 2005Yoav Y. Schechner, Nir Karpel 2005 Best polarization state Worst polarization state Best polarization state Recovered image [Left] The raw images taken through a polarizer. [Right] White-balanced results: The recovered image is much clearer, especially at distant objects, than the raw image
  76. 76. Varying PolarizationVarying Polarization • Schechner, Narasimhan, NayarSchechner, Narasimhan, Nayar • Instant dehazingInstant dehazing of images usingof images using polarizationpolarization
  77. 77. Spatial Augmented Reality | Raskar 2011 Pamplona , Mohan, Oliveira, Raskar, Siggraph 2010 NETRA: Near Eye Tool for Refractive Assessment EyeNetra.com
  78. 78. 90 Confocal Microscopy Examples Slides by Doug Lanman
  79. 79. Beyond Visible SpectrumBeyond Visible Spectrum CedipRedShift
  80. 80. MIT Media LabMIT Media Lab Camera CultureCamera Culture Ramesh RaskarRamesh Raskar MIT Media LabMIT Media Lab http:// CameraCulture . info/http:// CameraCulture . info/ Computational Camera &Computational Camera & Photography:Photography: Computational Camera &Computational Camera & Photography:Photography:
  81. 81. http://www.flickr.com/photos/pgoyette/107849943/in/photostream/
  82. 82. ScheimpflugScheimpflug principleprinciple
  83. 83. Ramesh Raskar, Computational Illumination Computational Illumination
  84. 84. Edgerton 1930’sEdgerton 1930’s Multi-flash Sequential Photography Stroboscope (Electronic Flash) Shutter Open Flash Time
  85. 85. Ramesh Raskar, Karhan Tan, Rogerio Feris, Jingyi Yu, Matthew Turk Mitsubishi Electric Research Labs (MERL), Cambridge, MA U of California at Santa Barbara U of North Carolina at Chapel Hill Non-photorealistic Camera:Non-photorealistic Camera: Depth Edge DetectionDepth Edge Detection andand StylizedStylized RenderingRendering usingusing Multi-Flash ImagingMulti-Flash Imaging
  86. 86. Depth Edges
  87. 87. Our MethodCanny
  88. 88. Flash MattingFlash Matting Flash Matting, Jian Sun, Yin Li, Sing Bing Kang, and Heung-Yeung Shum, Siggraph 2006
  89. 89. DARPA Grand ChallengeDARPA Grand Challenge
  90. 90. The Media Lab Camera Culture Epsilon Photography Capture multiple photos, each with slightly different camera parameters. • Exposure settings • Spectrum/color settings • Focus settings • Camera position • Scene illumination
  91. 91. The Media Lab Camera Culture Lens Sensor Camera Static Scene Image Destabilization [Mohan, Lanman et al. 2009]
  92. 92. The Media Lab Camera Culture Static Scene Lens Motion Sensor Motion Camera Image Destabilization [Mohan, Lanman et al. 2009]
  93. 93. MIT Media Lab Camera Culture Our Prototype
  94. 94. MIT Media Lab Camera Culture Adjusting the Focus Plane all-in-focus pinhole image
  95. 95. MIT Media Lab Camera Culture Defocus Exaggeration destabilization simulates a reduced f-number
  96. 96. The Media Lab Camera Culture Capturing Gigapixel Images [Kopf et al, 2007] 3,600,000,000 Pixels Created from about 800 8 MegaPixel Images
  97. 97. The Media Lab Camera Culture Capturing Gigapixel Images [Kopf et al, 2007]
  98. 98. Color Original Grayscale New Method Color2Gray:Color2Gray: Salience-PreservingSalience-Preserving Color RemovalColor Removal SIGGRAPH 2005 Gooch, Olsen, Tumblin, Gooch

×