1. The document discusses camera culture and computational photography led by Ramesh Raskar at the MIT Media Lab. 2. It outlines Raskar's vision of using emerging technologies to better capture and share visual information through new imaging platforms. 3. Some goals include giving consumers superhuman vision, seeing inside the body for health, and putting the photographer back in the photo.

1. Camera Culture Ramesh Raskar Camera Culture MIT Media Lab http://raskar.info http://cameraculture.info Ramesh Raskar Associate Professor Future of Imaging

2. Film-like Digital Photography

4. Cameras Phones are Everywhere Kush R. Varshney, “ Jagannath Temple ,” Puri, Orissa, India, Dec. 2007.

5. Wish List Today Consumers Super-human vision Microscope like resolution High speed (burst, video, no blur) See inside the body (health) Auto-trigger Battery life Zero start or shutter delay Keep only ‘good’ pics Find ‘relevant’ pics and better archiving/access Put photographer back in photo! Companies Cost Resolution Low-light sensitivity, HDR ( For cameraphone ) Size and depth Stereo and 3D Mecha-free zoom/focus Auto-tagging for sharing Recognition

6. Computational Photography Computational Illumination Computational Camera Scene : 8D Ray Modulator Display Generalized Sensor Generalized Optics Processing Ray Bender Ray Sampler Ray Reconstruction Generalized Optics Recreate 4D Lightfield Light Sources Modulators 4D Incident Lighting 4D Light Field

7. Can you look around a corner ?

9. Convert LCD into a big flat camera ?

11. Fernald, Science [Sept 2006] Shadow Refractive Reflective

12. Mimicking the Human Eye Lens Detector Pixels Image Slide by Shree Nayar Reproduce for the eye

13. Where are the ‘camera’s?

14. Where are the ‘camera’s?

15. We focus on creating tools to better capture and share visual information via a new class of imaging platforms Ramesh Raskar Ramesh Raskar http://raskar.info

16. What is the Media Lab? A Graduate Program in the Media Arts & Sciences Houses ~150 students and 30 PIs A Research Lab that spans across disciplines and academic/ industrial lines 65 sponsor companies Sponsors get free, non-exclusive licenses for ML IP Founded in 1985 by Nicholas Negroponte and Jerome Wiesner

17. Media Lab Vision Biomechatronics Neuroengineering Smart Fabrics Rethinking Cameras Human 2.0 Multimedia Ubiquitous Computing Social Media Computer Vision Sensor Networks Software Agents Bits Atoms People 1990s Body Brain Technology 2007+ Publishing Broadcast Computer 1980s

18. Close Ties With Industry 03/08/10 R E S E A R C H S T R A T E G Y Our 65 corporate sponsors include some of the most creative companies in the world

19. Center for Future Storytelling Launched with 7-year, $25M funding with Plymouth Rock Studios Opportunities for participation by other organizations Satellite lab facility to open at Plymouth Rock in 2010 3 Co-Directors Michael Bove (Object Based Mediaand Displays) Cynthia Breazeal (Sociable Robots) Ramesh Raskar (Cameras, Performance Capture) + Technical/Creative Advisor (Glorianna Davenport) Making stories more interactive, improvisational, social Managing the tension between the “master storyteller” and a dynamic, pervasive viewing/listening environment Developing new production, distribution, and display technologies MIT Media Laboratory

20. Smart Cities What if cars could stack like shopping carts in cities? Ryan Chin and Bill Mitchell

21. ADAPTABILTIY Hyper - Adaptability Bio- Mechatronics Music/Mind/ Health People - Sense Sociable Robots Neuro- Media HUMAN ADAPTABILITY Bio- Mechatronics Neuro- Media Sociable Robots People - Sense Music/Mind/ Health

22. Camera Culture Ramesh Raskar Ramesh Raskar Associate Professor, MIT Media Lab http://raskar.info New Emerging Technologies Medical Imaging Entertainment User Interfaces Industrial Vision The impact of Next Billion Cameras Movie-making, news reporting Social stability Visual Social Computing Image-based commerce Future of Imaging

23. Medical Imaging Today .. http://info.med.yale.edu/intmed/cardio/imaging/techniques/ct_imaging/

24. Self Evaluation of Eye

26. Self Evaluation of Eye

30. What are the problems with ‘real’ photo in conveying information ? Why do we hire artists to draw what can be photographed ?

31. Shadows Clutter Many Colors Highlight Shape Edges Mark moving parts Basic colors

32. Cartoon Camera Using Depth Edges

37. Canny Intensity Edge Detection Our Method Photo Result

39. Convert LCD into a big flat camera ? Beyond Multi-touch: 3D Gestures

40. Large Virtual Camera for 3D Interactive HCI and Video Conferencing Matthew Hirsch, Henry Holtzman Doug Lanman, Ramesh Raskar Siggraph Asia 2009 BiDi Screen

41. Touch + Hover using Thin, Depth Sensing LCD Sensor

42. Sensing Depth from Array of Virtual Eyes in LCD

43. Photos of tomorrow: computed not recorded http://scalarmotion.wordpress.com/2009/03/15/propeller-image-aliasing/

44. Synthesis Low Level Mid Level High Level Hyper realism Raw Angle, spectrum aware Non-visual Data, GPS Metadata Priors Comprehensive 8D reflectance field Computational Photography Digital Epsilon Coded Essence Computational Photography aims to make progress on both axis Camera Array HDR, FoV Focal stack Decomposition problems Depth Spectrum LightFields Human Stereo Vision Transient Imaging Virtual Object Insertion Relighting Augmented Human Experience Material editing from single photo Scene completion from photos Motion Magnification Phototourism Resolution

45. Camera Culture Ramesh Raskar Understand the World Identify/recognize Objects 3D Awareness Interact with embedded information

46. Can you ‘see’ around a corner ?

47. Femto-Photography: Higher Dimensional Capture FemtoFlash UltraFast Detector Computational Optics Serious Sync

48. Application: Rescue and Planning

50. Bokode

52. Defocus blur of Bokode

53. Coding in Angle Mohan, Woo, Smithwick, Hiura, Raskar [Siggraph 2009]

54. circle of confusion  circle of information camera Bokode (angle) Quote suggested by Kurt Akeley Encoding in Angle , not space, time or wavelength sensor

55. Product labels Street-view Tagging

56. cell-phone camera close to the Bokode (10,000+ bytes of data)

57. Camera Culture Ramesh Raskar Understand the World Identify/recognize Objects 3D Awareness Interact with embedded information

58. Visual Computing Forward (Synthesize) Special Effects, Image Processing, Design Inverse (Understand) Human-like performance Recognizing/finding objects, image search Challenging !

59. Power of the Processor Power of the Network Power of People 1985 1995 2005 2015 Powershifts

60. Crowdsourcing http://www.wired.com/wired/archive/14.06/crowds.html Amazon Mechanical Turk: Steve Fossett search ReCAPTCHA=OCR

61. Visual Social Computing Image-based commerce By the people, for the people, of the people Next Trend in India ? Harnessing human power beyond language barriers Business Model ? Outsource -> Image-based Crowdsource

62. Cameras in Developing Countries http://news.bbc.co.uk/2/hi/south_asia/7147796.stm Community news program run by village women

63. Developing Countries: CAMForms Paper forms with barcodes 83-bit 2D codes (including seven bits of error correction) Parikh (2005)

64. User Generated Content Visualization Google Map overlayed with geo-tagged photos Image-based Mashups http://phototour.cs.washington.edu/ Photo Synth

65. Socio-Political Goals: B’Tselem From websiteof : Israeli Information Center for Human Rights in the Occupied Territories

66. Truth in Images From Hany Farid LA Times March’03

67. Vision thru tongue http://www.pbs.org/kcet/wiredscience/story/97-mixed_feelings.html Solutions for the Visually Challenged http://www.seeingwithsound.com/

68. Camera Culture Ramesh Raskar How will the next billion cameras change the social culture ? How can we augment the camera to support best ‘image search’ ? How will camera improve trust and social stability ? How will movie-making, news reporting change ? Next model for image-based commerce ? Visual Social Computing

69. MIT Media Lab Workshops Session before lunch (12:15 to 1pm): How to Invent? Health, Entertainment and User Interfaces Session after lunch: (by Invitation Only) How to innovate Brainstorming

70. Camera Culture Ramesh Raskar Ramesh Raskar Associate Professor, MIT Media Lab http://raskar.info New Emerging Technologies Medical Imaging Entertainment User Interfaces Industrial Vision The impact of Next Billion Cameras Movie-making, news reporting Social stability Visual Social Computing Image-based commerce Future of Imaging

71. END Part 1

72. Start Part 2

73. Camera Culture Ramesh Raskar Camera Culture MIT Media Lab http://raskar.info http://cameraculture.info Ramesh Raskar Associate Professor

74. After X , what is ne X t How to Invent? Ramesh Raskar, MIT Media Lab

75. X d X++ X X+Y X X ne X t Ramesh Raskar, MIT Media Lab

76. Simple Exercise .. Image Compression Save Bandwidth and storage What is ne X t

77. Strategy #1: X d Extend it to next dimension (or some other) dimension Flickr to Youtube Wikipedia to .. ? Text, Audio (Speech), Image, Video .. Whats next ? CD .. Images to infrared, sound, ultrasound Macro scale to microscale Airbag for car to airbag for .. ?

78. Strategy #2: X+Y Fusion of the dissimilar More dissimilar, more spectacular the output Example Scientific imaging + Photography Coded aperture Tomography

79. Prototype camera 4000 × 4000 pixels ÷ 292 × 292 lenses = 14 × 14 pixels per lens Contax medium format camera Kodak 16-megapixel sensor Adaptive Optics microlens array 125 μ square-sided microlenses

80. Example of digital refocusing

81. Imaging in Sciences: Computer Tomography http://info.med.yale.edu/intmed/cardio/imaging/techniques/ct_imaging/

82. Self Evaluation of Eye

84. Strategy #3: X Do exactly the opposite Processing, Memory, Bandwidth In Computing world, in any era, one of this is a bottleneck But overtime, they change. You can often take an older idea and do exactly the opposite. E.g. bandwidth is now considered virtually limitless Business Process Reengineering (BPR) Michael Hammer, James Champy, 1990s In imaging: SLR: Faster mirror flip or no mirror flip Companies spent years improving mirror flip speed Why not just remove it? More computation Less light

85. Toll Free calls Reverse Auction

86. Power of the Processor Power of the Network Power of People 1985 1995 2005 2015 Powershifts

87. Larval Trematode Worm

88. Strategy #4: X Given a Hammer .. Find all the nails Sometimes even screws and bolts Given a cool solution/technique/Opportunity Find other problems (Where to find them?) Examples Peltier effect: Create a jacket that keeps you warm or cold Mobile phone opportunity

89. Strategy #5: X Given a nail, Find all hammers Sometimes even screwdrivers and pliers may work Given a problem, Find other solutions (Where to find them?) Examples App store (Apple) .. Open platform for all devices ..

90. Strategy #6: X++ Pick your adjective .. Making it faster, better, cheaper neXt = adjective + X

91. X++ : Add your favorite adjective Context aware, Adaptive (temporally) Coherent, Hierarchical, Progressive Efficient Parallelized Distributed Good example: Image or video compression schemes

92. X++ : Add your favorite adjective Good example: Image or video compression schemes But X++ is a sign The field is maturing in terms of research but booming in business impact Kaizen Small incremental changes Japanese Management styles (6sigma, Kanban) Mainly to save money/time/resources. Not everyone can do it. GM, 0.84 suggestions per employee vs Toyota 18. GM accepted 23%, Toyota 90%

93. Hexagon Corners for Different Stages [Bruce Tuckman, 1965] Forming Storming Norming Performing Adjourning

94. Pitfalls These six ways are only a start They are a good mental exercise and will allow you to train as a researcher Great for projects But Maynot produce radically new ideas Sometimes a danger of being labeled incremental Could be into ‘public domain ideas’

95. What are Bad ideas to pursue X then Y (then Z) X+Y is great with true fusion, fusion of dissimilar is best But avoid a ‘pipeline’ systems, where the output of one is THEN channeled into the input of the next stage, and non of the components are novel Follow the hype (too much competition) Do because it can be done (Why do we climb a mountain? because it is there! ) But only the first one gets a credit. May make you strong, and give you a sense of achievement but not a research project.

96. X d X++ X X+Y X X ne X t Ramesh Raskar, MIT Media Lab

97. End Part 2

98. Camera Culture Ramesh Raskar Camera Culture MIT Media Lab http://raskar.info http://cameraculture.info

99. Extra Slides

100. CREATIVITY Civic Media Next Billion Network Siftables Scratch Hyper - Creativity High- Low Tech CREATIVE ANARCHY Civic Media Next Billion Network Siftables Scratch High- Low Tech

101. AWARENESS New Media Medicine Smart Cities Sense-able Societies Hyper- Awareness Human Speechome X-Reality TOTAL AWARENESS Human Speechome Sense-able Societies X-Reality Smart Cities New Media Medicine

102. Today a billion plus people enjoy the benefits of a digital lifestyle BUT The deep impact of technology on individuals, society and business lies just ahead. Frank Moss, MIT Media Lab

103. Future of the Individual Future of Society Future of Business

104. Future of the Individual Future of Society Future of Business