• Like
Link to SIGGRAPH PPT presentation
Upcoming SlideShare
Loading in...5
×

Thanks for flagging this SlideShare!

Oops! An error has occurred.

Link to SIGGRAPH PPT presentation

  • 714 views
Published

 

Published in Business , Technology
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Be the first to comment
No Downloads

Views

Total Views
714
On SlideShare
0
From Embeds
0
Number of Embeds
0

Actions

Shares
Downloads
2
Comments
0
Likes
1

Embeds 0

No embeds

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
    No notes for slide

Transcript

  • 1. Ramesh Raskar, Paul Beardsley, Jeroen van Baar, Yao Wang, Paul Dietz, Johnny Lee, Darren Leigh, Thomas Willwacher Mitsubishi Electric Research Labs (MERL), Cambridge, MA R F I G Lamps : Interacting with a Self-describing World via Photosensing Wireless Tags and Projectors
  • 2. Radio Frequency Identification Tags (RFID) microchip Antenna No batteries, Small size, Cost few cents
  • 3. Warehousing Routing Library Baggage handling Currency Livestock tracking
  • 4. Conventional Passive RFID
  • 5. Tagged Books in a Library
    • Id
    • List of books in RF range
    • No Precise Location Data
    • Are books in sorted order ?
    • Which book is upside down ?
  • 6. Where are boxes with Products close to Expiry Date ?
  • 7. Conventional RF tag Photo-sensing RF tag
  • 8. Photosensor ? Compatible with RFID size and power needs Projector ? Directional transfer, AR with Image overlay
  • 9. b . Tags respond with date and precise (x,y) pixel location, Projector beams ‘Ok’ at that location a . Photosensing RFID tags are queried via RF c . Multiple users can simultaneously work from a distance without RF collision
  • 10.  
  • 11. RFID (Radio Frequency Identification) RFI G (Radio Frequency Id and Geometry )
  • 12. Prototype Tag RF tag + photosensor
  • 13. Outline
    • Photosensing RF tags
      • Location sensing
    • Geometric Operations
      • Multi-view analysis
      • Deformation
    • Interactive Projection
      • Mouse-like interaction
      • Image stabilization
  • 14. Outline
    • Photosensing RF tags
      • Location sensing
    • Geometric Operations
      • Multi-view analysis
      • Deformation
    • Interactive Projection
      • Mouse-like interaction
      • Image stabilization
    • Applications
  • 15. Pattern MSB Pattern MSB-1 Pattern LSB Projected Sequential Frames
    • Handheld Projector beams binary coded stripes
    • Tags decode temporal code
  • 16. Pattern MSB Pattern MSB-1 Pattern LSB Projected Sequential Frames
    • Handheld Projector beams binary coded stripes
    • Tags decode temporal code
  • 17. Pattern MSB Pattern MSB-1 Pattern LSB Projected Sequential Frames
    • Handheld Projector beams binary coded stripes
    • Tags decode temporal code
  • 18. Pattern MSB Pattern MSB-1 Pattern LSB Projected Sequential Frames
    • Handheld Projector beams binary coded stripes
    • Tags decode temporal code
  • 19. Pattern MSB Pattern MSB-1 Pattern LSB Projected Sequential Frames
    • Handheld Projector beams binary coded stripes
    • Tags decode temporal code
  • 20. Pattern MSB Pattern MSB-1 Pattern LSB
    • For each tag
    • From light sequence, decode x and y coordinate
    • Transmit back to RF reader ( Id , x, y )
    0 1 1 0 0 X=12
  • 21. Visual feedback of 2D position
    • Receive via RF {( x 1 ,y 1 ), ( x 2 ,y 2 ), …} pixels
    • Illuminate those positions
  • 22. Limitations
    • Line of sight
      • Surface patches
      • Multiple tags per object
    • Ambient light
      • Modulated infrared
    • Surface reflectance and shadows
  • 23. Outline
    • Photosensing RF tags
      • Location sensing
    • Geometric Operations
      • Multi-view analysis
      • Deformation
    • Interactive Projection
      • Mouse-like interaction
      • Image stabilization
    • Applications
  • 24. Find 3D coordinates
    • Observe structured patterns via Camera
    • Triangulate cam/proj view
  • 25. 3D from 2 Projector Views (Structure from Motion)
    • Two+ unknown projector views
    • Correspondence is trivial
    • Applications
      • Detect 3D deformations
      • Trajectory grouping
  • 26. Change Detection without fixed camera, in any lighting condition Compare with new coordinates from a different view Record coordinates of tags from one view Before After
  • 27. Texture Adaptation
  • 28. Outline
    • Photosensing RF tags
      • Location sensing
    • Geometric Operations
      • Multi-view analysis
      • Deformation
    • Interactive Projection
      • Desktop-like interaction
      • Image stabilization
    • Applications
  • 29. Desktop-like Interaction Selecting tags
  • 30. Support for handheld projection
  • 31. Mouse Simulation
    • Cursor follows handheld projector motion
    • Pre-warped image remains stable
  • 32. Image Quasi-Stabilization Eliminate hand jitter using inertial sensors+camera
  • 33. Absolute Stabilization Image stays registered with world features
  • 34. Image Stabilization
  • 35. Interactive Projection (Also in Emerging Technologies Booth)
  • 36. Adaptive Projection ‘ Paste’ : Geometric and Photometric compensation
  • 37. Prototype Handheld Projector
  • 38. Related Work
    • Smart objects
      • Smart-its
      • FindIT Flashlight (Ma and Paradiso 2002)
    • Location sensing
      • Multiple readers, large antennas
      • Olivetti Active Badge , Xerox PARCtab (Want 1995)
    • Interaction and Display
      • Augmented Reality
      • Projector-based AR, Shaderlamps
        • (Raskar 1998, Underkoffler 1999, Pinhanez 2001)
      • Handheld projector (Raskar 2003)
  • 39. Applications
    • Single Tags
      • Authoring for AR, Store augmentation: display per object
    • Multiple tags on objects
      • 3D Pose
      • Packing and placement strategy
      • Robot navigation
    • Interaction between tagged objects
      • History of geometric interaction
      • Orientation mismatch
    • Distributed tags
      • Deformation
      • Interpolated values for temperature sensors
  • 40. Acknowledgements
    • MERL
      • Cliff Forlines
      • Joe Marks,
      • Dick Waters, Kent Wittenburg
      • Vlad Branzoi
      • Rebecca Xiong, Debbi van Baar
    • Mitsubishi Electric, Japan
      • Mamuro Kato,
      • Keiichi Shiotani
  • 41. Interacting with a Self-Describing World
    • Hybrid optical and RF communication
      • Photosensing Wireless Tags + Projector illumination
    • Geometric Analysis
      • Location, Selection, Augmentation At a distance
    • Interactive Projection
      • Desktop like interaction
      • Stabilized images
    • RFIG Applications beyond logistics
      • Identity, Geometry, History, Annotation
    www.MERL.com
  • 42. Find tag location using handheld Projector Photosensing Wireless Tags Many geometric ops R F I R F I D Interactive stabilized projection (Radio Frequency Id & Geometry ) G
  • 43. (End of Presentation)
  • 44. Towards Passive RFID
    • Photosensor
      • Most compatible with passive batteryless RFID
      • Power: near zero power for sensing
      • Size: Each photocell is tens of micrometer (smaller than RFID which are about 500 micrometer)
      • Very low cost, can be built with silicon of RFID microchip
      • RFID+Photosensor can be size of grain of rice
      • RFIG tag is ‘visible’ to projector but not to humans
      • Current prototype uses battery because passive tags are difficult to program
    • Other possibilities
      • Adding a light emitting diode (LED)
        • Requires on board power
        • Size has to be considerably large
        • Size and power requirements are high
  • 45. Future Directions
    • Sophisticated tags
    • Light modulation
    • Handheld projector
  • 46. Comparison with Bar-codes
    • RFID
      • Barcodes take up more space
      • Long bit sequences
      • Multiple tags can be simultaneously read
      • Read distance is high
    • RFIG
      • Multiple barcodes per object is cumbersome
      • Difficult to attach sensor (e.g. temperature)
  • 47. Locate Tags Precisely using a handheld scanner/projector
    • Handheld device
    • Eliminates multiple RF readers for triangulation or large antennas
    • Projector indicates location of tag
    Illuminate tags with binary coded stripes with projector
  • 48.
    • Photosensor added to RFID tags
    • Hybrid communication via RF and via light
    • Solve 3 main issues with conventional RFIDs
    • Locate tags precisely
    • Select a subset of tags for operation
    • Long distance operation without collision
    • Applications beyond inventory management
    • Warehouse: locate objects for specific queries
    • Training and maintenance: interactive augmented reality
    • Surveillance: keep trail of object locations
  • 49. 2. Select subset of tags for operation
    • Advantages
    • Only tags which are illuminated respond to queries
    • No RF collision between multiple tags
    • Projector also use as a display to indicate result of operation
    • Technique
    • Illuminate subset of tags with flashlight or projector
    3. Long distance operation without RF collision
  • 50. Futures
    • We are used to fixed displays like desktops, laptops, and static projectors.
    • Handheld devices with screens have become common over the last decade – but the displays are small.
    • Handheld projectors offer a new display modality with its own characteristics - portable device size but with a large display.
    • Add interaction to achieve ‘the world as a desktop’…
  • 51. Interactive Projection
    • Desktop like interaction
      • Cursor follows use motion
      • Center pixel is the pointer
      • Rest of the image is stabilized
      • Selection
    • Image stabilization
      • Absolute: camera and inertial sensor
      • Relative: world markers
    • Copy-paste
      • Camera captures texture during ‘copy’
      • Projection after geometric/photometric adaptation