Raskar 6Sight Keynote Talk Nov09

Camera Culture Ramesh Raskar Camera Culture Associate Professor, MIT Media Lab Photography Wish List http://raskar.info

Film-like Digital Photography

Fernald, Science [Sept 2006] Shadow Refractive Reflective

‘ film-like’ Photography Lens Detector Pixels Image Slide by Shree Nayar Reproduce for the eye

Wish List Today
Consumers
Super-human vision
Microscope like resolution
High speed (burst, video, no blur)
See inside the body (health)
Auto-trigger
Battery life
Zero start or shutter delay
Keep only ‘good’ pics
Find ‘relevant’ pics and better archiving/access
Put photographer back in photo!
Companies
Cost
Resolution
Low-light sensitivity, HDR
( For cameraphone ) Size and depth
Stereo and 3D
Mecha-free zoom/focus
Auto-tagging for sharing
Recognition

Computational Photography Computational Illumination Computational Camera Scene : 8D Ray Modulator Display Generalized Sensor Generalized Optics Processing Ray Bender Ray Sampler Ray Reconstruction Generalized Optics Recreate 4D Lightfield Light Sources Modulators 4D Incident Lighting 4D Light Field

Can you look around a corner ?

Computational Photography [Raskar and Tumblin]
Epsilon Photography
Low-level vision: Pixels
‘ Ultimate camera’
Coded Photography
Mid-Level Cues:
Regions, Edges, Motion, Direct/global
‘ Scene analysis’
Essence Photography
High-level understanding
‘ New artform’
captures a machine-readable representation of our world to hyper-realistically synthesize the essence of our visual experience.

Camera Culture Ramesh Raskar Camera Culture Associate Professor, MIT Media Lab Computational Photography Wish List http://raskar.info

Synthesis Low Level Mid Level High Level Hyper realism Raw Angle, spectrum aware Non-visual Data, GPS Metadata Priors Comprehensive 8D reflectance field Computational Photography Digital Epsilon Coded Essence Computational Photography aims to make progress on both axis Camera Array HDR, FoV Focal stack Decomposition problems Depth Spectrum LightFields Human Stereo Vision Transient Imaging Virtual Object Insertion Relighting Augmented Human Experience Material editing from single photo Scene completion from photos Motion Magnification Phototourism Resolution

Camera Culture Ramesh Raskar Wish #1 Ultimate Post-capture Control

Digital Refocusing using Light Field Camera 125 μ square-sided microlenses [Ng et al 2005]

Zooming into the raw photo

Mask based Light Field Camera [Veeraraghavan, Raskar, Agrawal, Tumblin, Mohan, Siggraph 2007 ] Mask Sensor

Captured 2D Photo

x y u v 121 Sub-aperture Views

Samples individual rays
Predefined spectrum for lenses
Chromatic abberration
High alignment precision
For each lenslet, peripheral pixels are wasted
Negligible Light Loss
Samples coded combination of rays
Supports any wavelength
Reconfigurable f/# , Easier alignment
No wastage
High resolution image for parts of scene in focus
50 % Light Loss due to mask
Microlens array Sensor Plenoptic Camera Heterodyne Camera Mask Sensor

Figure 2 results Input Image Motion Blur in Low Light

Motion Blur in Low Light

Fluttered Shutter Camera Raskar, Agrawal, Tumblin Siggraph2006 Ferroelectric shutter in front of the lens is turned opaque or transparent in a rapid binary sequence

Image Deblurred by solving a linear system. No post-processing Blurred Taxi

Motion Blur in Low Light

Compact Programmable Lights ?

Image-Based Re-lighting Measure incoming light in Milan, Light the actress in LA Matte the background Matched LA and Milan lighting. Debevec et al., SIGG2001

Camera Culture Ramesh Raskar
Wish #1
Ultimate Post-capture Control
Digital Refocus and Motion blur
Emulate studio light from compact flash

Camera Culture Ramesh Raskar Wish #2 Freedom from Form

Convert LCD into a big flat camera ? Beyond Multi-touch: 3D Gestures

Large Virtual Camera for 3D Interactive HCI and Video Conferencing Matthew Hirsch, Henry Holtzman Doug Lanman, Ramesh Raskar Siggraph Asia 2009 BiDi Screen

Touch + Hover using Thin, Depth Sensing LCD Sensor

Sensing Depth from Array of Virtual Cameras in LCD

Shallow DoF with Simple Lens Lots of glass; Heavy; Bulky; Expensive

Image Destabilization: Programmable Defocus using Lens and Sensor Motion Ankit Mohan, Douglas Lanman, Shinsaku Hiura, Ramesh Raskar MIT Media Lab MIT Media Lab Camera Culture

Image De stabilization Lens Sensor Camera Static Scene

Image Destabilization Static Scene Lens Motion Sensor Motion Camera Mohan, Lanman,Hiura, Raskar ICCP 2009

Shifting Pinhole and Sensor A B A’ B’ Pinhole v p Sensor v s d a d b d s Focus Here

Shifting Pinhole and Sensor A B Pinhole A’ B’ v p Sensor v s d a d b d s Focus Here

“ Time Lens” Ratio of speeds Lens Equation: Virtual Focal Length: Virtual F-Number:

Time Lens:

Small Aperture Photo all-in-focus

Destabilized Small Aperture focused in the front using destabilization

Adjusting the Focus Plane focused in the middle using destabilization

Adjusting the Focus Plane focused in the back using destabilization

Camera Culture Ramesh Raskar Wish #3 Understand the World

Real or Fake ?

Direct Reflection

Fake Real Internal Reflection

High Frequency Pattern surface Camera Flash Nayar, Krishnan, Grossberg, Raskar [Siggraph 2006] i

Convert single 2D photo into 3D ? Snavely, Seitz, Szeliski U of Washington/Microsoft: Photosynth

Exploit Community Photo Collections U of Washington/Microsoft: Photosynth

Can you look around a corner ?

Can you ‘see’ around a corner ?

Femto-Photography: Higher Dimensional Capture FemtoFlash UltraFast Detector Computational Optics Serious Sync

Bokode

Defocus blur of Bokode

Coding in Angle Mohan, Woo, Smithwick, Hiura, Raskar [Siggraph 2009]

circle of confusion  circle of information
camera Bokode (angle) Quote suggested by Kurt Akeley Encoding in Angle , not space, time or wavelength sensor

Product labels
Street-view Tagging

capturing Bokodes cell-phone camera close to the Bokode (10,000+ bytes of data)

Wish #3
Understand the World
Identify/recognize Materials
3D Awareness
Interact with information

Camera Culture Ramesh Raskar Wish #4 Sharing Visual Experience

Wish #4
Sharing Visual Experience
LifeLog Auto-summary
Privacy in public and authentication
Great Photo-frames

6D Photo Frames One Pixel of a 6D Display = 4D Display Single Pixel of 6D Frame 1 2 1 1 2D 2D 2D

6D Photo Frames: Respond to Viewpoint + Ambient Light

Wish #4
Sharing Visual Experience
LifeLog Auto-summary
Privacy in public and authentication
Hyper-real Photo Frames
Print ‘material’

Camera Culture Ramesh Raskar Wish #5 Capturing Essence

Essence Photography
Beyond physical realism
New Visual Art Forms
Yu, McMillan

What are the problems with ‘real’ photo in conveying information ? Why do we hire artists to draw what can be photographed ?

Shadows Clutter Many Colors Highlight Shape Edges Mark moving parts Basic colors

Depth Edges with MultiFlash Raskar, Tan, Feris, Jingyi Yu, Turk – ACM SIGGRAPH 2004

Depth Discontinuities Internal and external Shape boundaries, Occluding contour, Silhouettes

Depth Edges

Our Method Canny

Canny Intensity Edge Detection Our Method Photo Result

Blind Camera Sascha Pohflepp, U of the Art, Berlin, 2006

Scene Completion Using Millions of Photographs Hays and Efros, Siggraph 2007

Camera Culture Ramesh Raskar Wish #5 …………………

Photos of tomorrow: computed not recorded http://scalarmotion.wordpress.com/2009/03/15/propeller-image-aliasing/

Questions
What will a camera look like in 10,20 years?
How will a billion networked and portable cameras change the social culture?
How will online photo collections transform visual social computing?
How will movie making/new reporting change?
computational-journalism.com

2 nd International Conference on Computational Photography Papers due November 2, 2009 http://cameraculture.media.mit.edu/iccp10

Ramesh Raskar and Jack Tumblin
Book Publishers: A K Peters
ComputationalPhotography.org

Post-capture control
Emulate studio lights with compact flash
Focus and motion blur
New forms
Flat camera, large LCDs as cameras
Image destabilization for larger aperture
Understand the world
Real or fake
Place 2D photo into 3D
Look around corner
Bokode: long distance barcode
Sharing
Lifelogs auto summary
Privacy/Verification
6D photoframes
Essence
New visual arts
Multi-flash camera
Delta-camera and Blind-camera
Camera Culture Group, MIT Media Lab Ramesh Raskar http://raskar.info Computational Photography Wish List Sensor

http://stellar.mit.edu	339
http://www.maximizingprogress.org	311
https://stellar.mit.edu	222
http://www.slideshare.net	14
http://www.google.com	1

Raskar 6Sight Keynote Talk Nov09

by MIT Media Lab, Camera Culture group on Nov 12, 2009

Accessibility

Categories

Tags

Upload Details

Usage Rights

5 Embeds 887

Statistics

Raskar 6Sight Keynote Talk Nov09 — Presentation Transcript