Your SlideShare is downloading. ×
0
Raskar Graphics Interface May05 Web
Raskar Graphics Interface May05 Web
Raskar Graphics Interface May05 Web
Raskar Graphics Interface May05 Web
Raskar Graphics Interface May05 Web
Raskar Graphics Interface May05 Web
Raskar Graphics Interface May05 Web
Raskar Graphics Interface May05 Web
Raskar Graphics Interface May05 Web
Raskar Graphics Interface May05 Web
Raskar Graphics Interface May05 Web
Raskar Graphics Interface May05 Web
Raskar Graphics Interface May05 Web
Raskar Graphics Interface May05 Web
Raskar Graphics Interface May05 Web
Raskar Graphics Interface May05 Web
Raskar Graphics Interface May05 Web
Raskar Graphics Interface May05 Web
Raskar Graphics Interface May05 Web
Raskar Graphics Interface May05 Web
Raskar Graphics Interface May05 Web
Raskar Graphics Interface May05 Web
Raskar Graphics Interface May05 Web
Raskar Graphics Interface May05 Web
Raskar Graphics Interface May05 Web
Raskar Graphics Interface May05 Web
Raskar Graphics Interface May05 Web
Raskar Graphics Interface May05 Web
Raskar Graphics Interface May05 Web
Raskar Graphics Interface May05 Web
Raskar Graphics Interface May05 Web
Raskar Graphics Interface May05 Web
Raskar Graphics Interface May05 Web
Raskar Graphics Interface May05 Web
Raskar Graphics Interface May05 Web
Raskar Graphics Interface May05 Web
Raskar Graphics Interface May05 Web
Raskar Graphics Interface May05 Web
Raskar Graphics Interface May05 Web
Raskar Graphics Interface May05 Web
Raskar Graphics Interface May05 Web
Raskar Graphics Interface May05 Web
Raskar Graphics Interface May05 Web
Raskar Graphics Interface May05 Web
Raskar Graphics Interface May05 Web
Raskar Graphics Interface May05 Web
Raskar Graphics Interface May05 Web
Raskar Graphics Interface May05 Web
Raskar Graphics Interface May05 Web
Raskar Graphics Interface May05 Web
Raskar Graphics Interface May05 Web
Raskar Graphics Interface May05 Web
Raskar Graphics Interface May05 Web
Raskar Graphics Interface May05 Web
Raskar Graphics Interface May05 Web
Raskar Graphics Interface May05 Web
Raskar Graphics Interface May05 Web
Raskar Graphics Interface May05 Web
Raskar Graphics Interface May05 Web
Raskar Graphics Interface May05 Web
Raskar Graphics Interface May05 Web
Raskar Graphics Interface May05 Web
Raskar Graphics Interface May05 Web
Raskar Graphics Interface May05 Web
Raskar Graphics Interface May05 Web
Raskar Graphics Interface May05 Web
Raskar Graphics Interface May05 Web
Raskar Graphics Interface May05 Web
Raskar Graphics Interface May05 Web
Raskar Graphics Interface May05 Web
Raskar Graphics Interface May05 Web
Raskar Graphics Interface May05 Web
Raskar Graphics Interface May05 Web
Raskar Graphics Interface May05 Web
Raskar Graphics Interface May05 Web
Raskar Graphics Interface May05 Web
Raskar Graphics Interface May05 Web
Upcoming SlideShare
Loading in...5
×

Thanks for flagging this SlideShare!

Oops! An error has occurred.

×
Saving this for later? Get the SlideShare app to save on your phone or tablet. Read anywhere, anytime – even offline.
Text the download link to your phone
Standard text messaging rates apply

Raskar Graphics Interface May05 Web

487

Published on

Published in: Technology, Business
0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total Views
487
On Slideshare
0
From Embeds
0
Number of Embeds
1
Actions
Shares
0
Downloads
12
Comments
0
Likes
0
Embeds 0
No embeds

Report content
Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
No notes for slide
  • Transcript

    • 1. Ramesh Raskar Mitsubishi Electric Research Labs (MERL) Cambridge, MA Special Effects in the Real World
    • 2. Special Effects <ul><ul><li>Photorealism </li></ul></ul><ul><ul><ul><li>Stays on screens </li></ul></ul></ul><ul><ul><ul><li>Yet to affect daily life in real time </li></ul></ul></ul><ul><ul><ul><ul><li>Entertainment and Visualization </li></ul></ul></ul></ul><ul><ul><ul><ul><li>Unlike other fields of CS </li></ul></ul></ul></ul>
    • 3. Changing Appearance Virtual light source
    • 4. Changing Virtual Illumination
    • 5. &nbsp;
    • 6. Special Effects in the Real World <ul><ul><li>Photorealism </li></ul></ul><ul><ul><ul><li>Limited to display screens </li></ul></ul></ul><ul><ul><ul><li>Yet to affect daily life in real time </li></ul></ul></ul><ul><ul><ul><ul><li>Entertainment and Visualization </li></ul></ul></ul></ul><ul><ul><ul><ul><li>Unlike other fields of CS </li></ul></ul></ul></ul><ul><ul><li>Fusion: real world with graphics </li></ul></ul><ul><ul><ul><li>Next big challenge in CG ? </li></ul></ul></ul><ul><ul><ul><li>Believable co-existence </li></ul></ul></ul>
    • 7. Changing Appearance Projector Projector Virtual light source
    • 8. Changing Virtual Illumination Raskar, Welch, Low, Bandyopadhyay, “ Shader Lamps ” (2000)
    • 9. Virtual Motion
    • 10. Virtual Motion Raskar, Ziegler, Willwacher, “Cartoon Dioramas in Motion,” (NPAR 2002)
    • 11. Dynamic Augmentation Projecting on Tracked Objects
    • 12. ShaderLamps Virtual Reflectance Virtual Illumination Interaction Virtual Motion www.ShaderLamps.com
    • 13. Projector-based Augmentation www.ShaderLamps.com Virtual Reflectance Virtual Illumination Interaction Virtual Motion
    • 14. Poor Man’s Palace
    • 15. Maya: World is an Illusion <ul><li>Fake World </li></ul><ul><ul><li>We all live in one </li></ul></ul><ul><li>Real-life Fakes </li></ul><ul><ul><li>Not just photos and videos but surroundings </li></ul></ul><ul><li>Privacy </li></ul>
    • 16. Augmented Reality (AR) With See-through Head-mounted Display © Andrei State
    • 17. Head-attached Body-attached spatial Spatial Augmented Reality Classification of Augmented Reality hand-held display spatial see-through display head- mounted display retinal display projector projector projector
    • 18. Being There <ul><li>Recreate Large Environments </li></ul><ul><ul><li>‘ BeingThere’, walk-around </li></ul></ul><ul><ul><li>Museums, Exhibitions, Training </li></ul></ul>Kok-Lim Low, Greg Welch, Anselmo Lastra, Henry Fuchs. “Life-Sized Projector-Based Dioramas,” Proc. ACM Symposium on Virtual Reality Software and Technology 2001 (VRST 2001), November 2001.
    • 19. Desired Virtual Model © Andrei State Projected Guidance for Placement
    • 20. &nbsp;
    • 21. Augmented Reality Head-mounted Display Projectors © Andrei State
    • 22. Opportunities <ul><ul><li>Information Augmentation </li></ul></ul><ul><ul><ul><li>Office, factory or assembly Training </li></ul></ul></ul><ul><ul><li>Increasing value of real products </li></ul></ul><ul><ul><ul><li>Smart illumination to enhance appearance of object </li></ul></ul></ul><ul><ul><li>Entertainment </li></ul></ul><ul><ul><ul><li>Animating objects, Museums, Special Effects, Laser Shows </li></ul></ul></ul><ul><ul><li>Advertising </li></ul></ul><ul><ul><ul><li>Car showrooms, Furniture store </li></ul></ul></ul><ul><ul><li>Design and Ergonomics </li></ul></ul><ul><ul><ul><li>Protyping products, visualize material changes </li></ul></ul></ul>Projector + Camera
    • 23. <ul><li>Why </li></ul><ul><ul><li>Special Effects in the Real World </li></ul></ul><ul><ul><li>Photorealism for everyday </li></ul></ul><ul><li>Tools for Augmentation </li></ul><ul><ul><li>Projectors, Sensors, RFID </li></ul></ul><ul><li>Illusion </li></ul><ul><ul><li>Beyond Visual Senses </li></ul></ul><ul><ul><li>Beyond Projectors </li></ul></ul><ul><ul><li>Beyond Gouraud </li></ul></ul>Spatially Augmented Reality
    • 24. Spatially Augmented Reality (SAR) HMD-VR Spatially Immersive-VR AR using HMD Spatially Augmented Reality e.g. CAVE Video or Optical see-through SAR, Shaderlamps
    • 25. AR Issues <ul><li>Preprocessing: </li></ul><ul><ul><li>Authoring </li></ul></ul><ul><li>Runtime: </li></ul><ul><ul><li>Identification : Recognition of objects </li></ul></ul><ul><ul><ul><li>Using markers and visual tags </li></ul></ul></ul><ul><ul><li>Registration : Finding relative pose of display device </li></ul></ul><ul><ul><ul><li>Dynamic estimate of translation and rotation </li></ul></ul></ul><ul><ul><ul><li>Render/Warp images </li></ul></ul></ul><ul><ul><li>Interaction: </li></ul></ul><ul><ul><ul><li>Widgets, Gesture recognition, Visual feedback </li></ul></ul></ul>
    • 26. Spatially Augmented Reality Raskar, vanBaar, Beardsley, Willwacher, Rao, Forlines ‘iLamps: Geometrically Aware and Self-Configurable Projectors’, SIGGRAPH 2003
    • 27. AR Issues <ul><li>Preprocessing: </li></ul><ul><ul><li>Authoring </li></ul></ul><ul><li>Runtime: </li></ul><ul><ul><li>Identification : Recognition of objects </li></ul></ul><ul><ul><ul><li>Using markers and visual tags </li></ul></ul></ul><ul><ul><li>Registration : Finding relative pose of display device </li></ul></ul><ul><ul><ul><li>Dynamic estimate of translation and rotation </li></ul></ul></ul><ul><ul><ul><li>Render/Warp images </li></ul></ul></ul><ul><ul><li>Interaction: </li></ul></ul><ul><ul><ul><li>Widgets, Gesture recognition, Visual feedback </li></ul></ul></ul><ul><ul><li>Photorealism ? </li></ul></ul>RFID Cameras Trackers
    • 28. Steps <ul><ul><li>Preprocessing </li></ul></ul><ul><ul><ul><li>Scan 3D object </li></ul></ul></ul><ul><ul><ul><li>Roughly align projectors </li></ul></ul></ul><ul><ul><ul><li>Compute correction </li></ul></ul></ul><ul><ul><li>Run time </li></ul></ul><ul><ul><ul><li>Render images of 3D model </li></ul></ul></ul><ul><ul><ul><li>Correct </li></ul></ul></ul>
    • 29. Steps <ul><ul><li>Preprocessing </li></ul></ul><ul><ul><ul><li>Scan 3D object and create virtual model </li></ul></ul></ul>Faro arm
    • 30. Steps <ul><ul><li>Preprocessing </li></ul></ul><ul><ul><ul><li>Scan 3D object and create model </li></ul></ul></ul><ul><ul><ul><li>Approximately position projector(s) </li></ul></ul></ul><ul><ul><ul><li>Calibrate by finding pose </li></ul></ul></ul>
    • 31. Steps <ul><ul><li>Preprocessing </li></ul></ul><ul><ul><ul><li>Scan 3D object and create model </li></ul></ul></ul><ul><ul><ul><li>Approximately position projector(s) </li></ul></ul></ul><ul><ul><ul><li>Calibrate by finding pose </li></ul></ul></ul><ul><ul><li>Run-time </li></ul></ul><ul><ul><ul><li>Inverse Rendering of 3D Models </li></ul></ul></ul>
    • 32. Warehousing Routing Library Baggage handling Currency Livestock tracking RFID
    • 33. Conventional Passive RFID
    • 34. Tagged Books in a Library <ul><li>Id : List of books in RF range </li></ul><ul><li>No Precise Location Data </li></ul><ul><li>Are books in sorted order ? </li></ul><ul><li>Which book is upside down ? </li></ul>
    • 35. Handheld Projector iLamps 2002 RFIG Lamps 2003-04 Pocket Projector 2004-05
    • 36. Mitsubishi Pocket Projector
    • 37. &nbsp;
    • 38. Prototype Tag RF tag + photosensor
    • 39. Conventional RF tag Photo-sensing RF tag
    • 40. &nbsp;
    • 41. Find tag location using handheld Projector Photosensing Wireless Tags Many geometric ops R F I R F I D Interactive stabilized projection (Radio Frequency Id &amp; Geometry) G Siggraph 2004
    • 42. AR with Photosensing RFID and Handheld Projector
    • 43. Pattern MSB Pattern MSB-1 Pattern LSB Projected Sequential Frames <ul><li>Handheld Projector beams binary coded stripes </li></ul><ul><li>Tags decode temporal code </li></ul>
    • 44. Pattern MSB Pattern MSB-1 Pattern LSB Projected Sequential Frames <ul><li>Handheld Projector beams binary coded stripes </li></ul><ul><li>Tags decode temporal code </li></ul>
    • 45. Pattern MSB Pattern MSB-1 Pattern LSB Projected Sequential Frames <ul><li>Handheld Projector beams binary coded stripes </li></ul><ul><li>Tags decode temporal code </li></ul>
    • 46. Pattern MSB Pattern MSB-1 Pattern LSB Projected Sequential Frames <ul><li>Handheld Projector beams binary coded stripes </li></ul><ul><li>Tags decode temporal code </li></ul>
    • 47. Pattern MSB Pattern MSB-1 Pattern LSB Projected Sequential Frames <ul><li>Handheld Projector beams binary coded stripes </li></ul><ul><li>Tags decode temporal code </li></ul>
    • 48. Pattern MSB Pattern MSB-1 Pattern LSB <ul><li>For each tag </li></ul><ul><li>From light sequence, decode x and y coordinate </li></ul><ul><li>Transmit back to RF reader (Id, x, y) </li></ul>0 1 1 0 0 X=12
    • 49. Visual feedback of 2D position <ul><li>Receive via RF {( x 1 ,y 1 ), ( x 2 ,y 2 ), …} pixels </li></ul><ul><li>Illuminate those positions </li></ul>
    • 50. RFID (Radio Frequency Identification) RFI G (Radio Frequency Id and Geometry )
    • 51. Projectors and Surfaces Curved Planar Non-planar Single Projector Multiple Projectors Pocket-Proj Objects 2000 2000 2002 2001 1999 2002 2003 1999 1998 Projector j User : T ?
    • 52. Pieces of the Puzzle Actuated Surfaces Actuated Workbench [Pangaro, Maynes-Aminzade, Ishii UIST 2002]
    • 53. <ul><li>Why </li></ul><ul><ul><li>Special Effects in the Real World </li></ul></ul><ul><ul><li>Photorealism for everyday </li></ul></ul><ul><li>Tools for Augmentation </li></ul><ul><ul><li>Projectors, Sensors, RFID </li></ul></ul><ul><li>Illusion </li></ul><ul><ul><li>Beyond Visual Senses </li></ul></ul><ul><ul><li>Beyond Projectors </li></ul></ul><ul><ul><li>Beyond Gouraud </li></ul></ul>Spatially Augmented Reality
    • 54. Believable Co-existence <ul><li>Other Senses </li></ul><ul><ul><li>Haptic, Smell, Taste, Locomotion </li></ul></ul><ul><li>Beyond Gouraud Effects </li></ul><ul><ul><li>Complex BRDF </li></ul></ul><ul><li>Display Choices </li></ul><ul><ul><li>Polymers, e-Ink </li></ul></ul>
    • 55. Augmentation of “five senses” <ul><li>Visual Displays </li></ul><ul><li>Auditory Displays </li></ul><ul><li>Haptic Displays </li></ul><ul><li>Locomotion Interfaces </li></ul><ul><li>Olfactory Displays </li></ul><ul><li>Taste Displays ? </li></ul><ul><li>Sixth Sense .. </li></ul>Extending sensory modalities
    • 56. Sense of Balance <ul><li>Galvanic Vestibular Stimulation </li></ul>Masahiko Inami (UEC)
    • 57. Electric Cutaneous Display Skin Nerve Electrode Dr. H. Kajimoto, U Tokyo Masahiko Inami (UEC)
    • 58. Smart Touch Masahiko Inami (UEC)
    • 59. Scent Display Yasuyuki Yanagida, ATR Spatio-Temporal Controlled Delivery of Scent
    • 60. Air Cannon Generating a doughnut-shaped vortex ring Aperture Push
    • 61. Air Cannon
    • 62. Prototype System <ul><li>Nose tracking </li></ul><ul><li>Multiple scents </li></ul><ul><li>Scent switching </li></ul><ul><li>Improved delivery range </li></ul>Yasuyuki Yanagida, ATR
    • 63. Beyond White Objects Grossberg, Nayar et al
    • 64. &nbsp;
    • 65. Grossberg, Nayar et al
    • 66. The Virtual Showcase Oliver Bimber Tracking and Mirrors
    • 67. Displays Contenders Organic LED Light Emitting Polymers E-Ink
    • 68. Holodust Perlin et al
    • 69. Fake World
    • 70. Beyond Gouraud Shading of White Objects <ul><li>Participating Media </li></ul><ul><ul><li>simulating or in presence of smoke, fog </li></ul></ul><ul><li>Complex Geometry </li></ul><ul><ul><li>Spaghetti </li></ul></ul><ul><li>Complex Reflectance </li></ul><ul><ul><li>Arbitrary appearance on arbitrary BRDF surfaces </li></ul></ul><ul><li>Shape and Motion </li></ul><ul><ul><li>Transformation and Animation of real surfaces </li></ul></ul><ul><li>NPR, Distortions, Perceptual factors </li></ul><ul><ul><li>Great thesis topics .. </li></ul></ul>
    • 71. Recap <ul><li>Special Effects in Real World </li></ul><ul><ul><li>Photorealism yet to impact daily life </li></ul></ul><ul><ul><li>Poor Man’s Palace </li></ul></ul><ul><li>Spatial Augmented Reality </li></ul><ul><ul><li>Solution for fusion </li></ul></ul><ul><ul><li>Geometry, Photometry, Id </li></ul></ul><ul><ul><li>Sense, Control, Compensate </li></ul></ul><ul><ul><li>Needs hardware components: Projectors, RFID, Sensors </li></ul></ul><ul><li>Open Problems </li></ul><ul><ul><li>Natural phenomenon, complex BRDF, other displays </li></ul></ul><ul><ul><li>All senses: haptic, olfactory, preprioception </li></ul></ul><ul><li>Next Challenge: Photorealism </li></ul>
    • 72. Spatial Augmented Reality <ul><li>Graphics </li></ul><ul><ul><li>Photorealism yet to impact daily life </li></ul></ul><ul><ul><li>Special effects in real world </li></ul></ul><ul><li>Spatial Augmentation </li></ul><ul><ul><li>Solution for fusion </li></ul></ul><ul><ul><li>Needs hardware components </li></ul></ul><ul><ul><li>Projectors, RFID, Sensors </li></ul></ul><ul><li>Next Challenge: Photorealism </li></ul>raskar.com
    • 73. More <ul><li>Networked fake world </li></ul><ul><li>Poor man’s palace </li></ul><ul><li>Intelligence moving from PC to Proj to Surf </li></ul><ul><li>Electronic items eventually become cheaper than those scavenged from nature </li></ul>
    • 74. Special Effects <ul><ul><li>Photorealism </li></ul></ul><ul><ul><ul><li>Limited to display screens </li></ul></ul></ul><ul><ul><ul><li>Does not affect daily life in real time </li></ul></ul></ul><ul><ul><ul><ul><li>Entertainment and Visualization </li></ul></ul></ul></ul><ul><ul><ul><ul><li>Unlike other fields of CS </li></ul></ul></ul></ul><ul><ul><ul><li>Provoke not offend </li></ul></ul></ul><ul><ul><ul><li>The primary advantage AR has over VR is that AR uses the parts of the body that aren&apos;t just the eyes and ears: proprioception, vestibular perception, and othe cues that old-fashioned VR just can&apos;t handle. The disjunct between vestibular information and visual information that you get in VR is the source of the motion-sickness that often accompanies it. </li></ul></ul></ul><ul><ul><ul><li>VR, like a lot of early &apos;cyberspace&apos; mythology, was built on an unrealistic rejection of the body, and a fantasy of &amp;quot;pure mind.&amp;quot; </li></ul></ul></ul><ul><ul><ul><li>Bud Fake Cup of Coffee 3000 </li></ul></ul></ul><ul><ul><ul><li>http://www.visit4info.com/details.cfm?adid=21038 </li></ul></ul></ul>
    • 75. Applications <ul><ul><li>Static setup </li></ul></ul><ul><ul><ul><li>Color everyday objects </li></ul></ul></ul><ul><ul><ul><li>Engineering drilling locations </li></ul></ul></ul><ul><ul><ul><li>Stage backdrops, seasons or aging </li></ul></ul></ul><ul><ul><li>Laser shows on buildings </li></ul></ul><ul><ul><li>Simulate motion </li></ul></ul><ul><ul><ul><li>Car shows, wheels, dirt marks </li></ul></ul></ul><ul><ul><ul><li>Virtual moving lights </li></ul></ul></ul>
    • 76. Applications Indoors, under controlled lighting <ul><ul><li>Architectural models </li></ul></ul><ul><ul><ul><li>Augment walk-around scaled model of buildings </li></ul></ul></ul><ul><ul><ul><li>Project and ‘paint’ surface colors, textures </li></ul></ul></ul><ul><ul><ul><li>Lighting, sunlight, seasons </li></ul></ul></ul><ul><ul><ul><li>Internal structure, pipes, wiring </li></ul></ul></ul><ul><ul><li>Assembly line </li></ul></ul><ul><ul><ul><li>Instructional text, images and procedures </li></ul></ul></ul><ul><ul><li>Entertainment </li></ul></ul><ul><ul><ul><li>Live shows, exhibits, demonstrations </li></ul></ul></ul>
    • 77. Tools for Augmentation <ul><li>Environment </li></ul><ul><ul><li>Sense, Compensate and Control </li></ul></ul><ul><ul><li>Id, Geometry, Photometry(reflectance/Illumination), Id </li></ul></ul><ul><li>Display </li></ul><ul><li>Rendering </li></ul>

    ×