Ramesh Raskar, Paul Beardsley, Jeroen van Baar, Yao Wang,  Paul Dietz, Johnny Lee, Darren Leigh, Thomas Willwacher   Mitsu...
Radio Frequency Identification Tags (RFID) microchip Antenna No batteries, Small size, Cost few cents
Warehousing Routing Library  Baggage  handling  Currency Livestock tracking
Conventional Passive RFID
Tagged Books in a Library <ul><li>Id   </li></ul><ul><li>List of books in RF range </li></ul><ul><li>No Precise Location D...
Where are boxes with  Products close to Expiry Date ?
Conventional RF tag Photo-sensing RF tag
Photosensor ? Compatible with RFID size and power needs Projector ? Directional transfer, AR with Image overlay
b . Tags respond with date and precise (x,y) pixel location, Projector beams ‘Ok’ at that location a . Photosensing RFID t...
 
RFID (Radio  Frequency  Identification) RFI G (Radio  Frequency  Id  and  Geometry )
Prototype Tag RF tag  +  photosensor
Outline <ul><li>Photosensing RF tags </li></ul><ul><ul><li>Location sensing </li></ul></ul><ul><li>Geometric Operations </...
Outline <ul><li>Photosensing RF tags </li></ul><ul><ul><li>Location sensing </li></ul></ul><ul><li>Geometric Operations </...
Pattern MSB Pattern MSB-1 Pattern LSB Projected Sequential Frames <ul><li>Handheld Projector beams binary coded stripes </...
Pattern MSB Pattern MSB-1 Pattern LSB Projected Sequential Frames <ul><li>Handheld Projector beams binary coded stripes </...
Pattern MSB Pattern MSB-1 Pattern LSB Projected Sequential Frames <ul><li>Handheld Projector beams binary coded stripes </...
Pattern MSB Pattern MSB-1 Pattern LSB Projected Sequential Frames <ul><li>Handheld Projector beams binary coded stripes </...
Pattern MSB Pattern MSB-1 Pattern LSB Projected Sequential Frames <ul><li>Handheld Projector beams binary coded stripes </...
Pattern MSB Pattern MSB-1 Pattern LSB <ul><li>For each tag </li></ul><ul><li>From light sequence, decode  x  and  y   coor...
Visual feedback of 2D position <ul><li>Receive via RF {( x 1 ,y 1 ), ( x 2 ,y 2 ), …} pixels </li></ul><ul><li>Illuminate ...
Limitations <ul><li>Line of sight </li></ul><ul><ul><li>Surface patches </li></ul></ul><ul><ul><li>Multiple tags per objec...
Outline <ul><li>Photosensing RF tags </li></ul><ul><ul><li>Location sensing </li></ul></ul><ul><li>Geometric Operations </...
Find 3D coordinates <ul><li>Observe structured patterns via Camera </li></ul><ul><li>Triangulate cam/proj view </li></ul>
3D from 2 Projector Views (Structure from Motion) <ul><li>Two+ unknown projector views </li></ul><ul><li>Correspondence is...
Change Detection  without fixed camera, in any lighting condition Compare with new coordinates from a different view   Rec...
Texture Adaptation
Outline <ul><li>Photosensing RF tags </li></ul><ul><ul><li>Location sensing </li></ul></ul><ul><li>Geometric Operations </...
Desktop-like Interaction Selecting tags
Support for handheld projection
Mouse Simulation <ul><li>Cursor follows handheld projector motion </li></ul><ul><li>Pre-warped image remains stable </li><...
Image Quasi-Stabilization Eliminate hand jitter using inertial sensors+camera
Absolute Stabilization Image stays registered with world features
Image Stabilization
Interactive Projection (Also in Emerging Technologies Booth)
Adaptive Projection ‘ Paste’ : Geometric and Photometric compensation
Prototype Handheld Projector
Related Work <ul><li>Smart objects </li></ul><ul><ul><li>Smart-its </li></ul></ul><ul><ul><li>FindIT Flashlight  (Ma and P...
Applications <ul><li>Single Tags </li></ul><ul><ul><li>Authoring for AR, Store augmentation: display per object </li></ul>...
Acknowledgements <ul><li>MERL </li></ul><ul><ul><li>Cliff Forlines </li></ul></ul><ul><ul><li>Joe Marks,  </li></ul></ul><...
Interacting with a Self-Describing World <ul><li>Hybrid optical and RF communication </li></ul><ul><ul><li>Photosensing Wi...
Find tag location using   handheld Projector Photosensing  Wireless Tags Many geometric ops  R F I  R F I D Interactive st...
(End of Presentation)
Towards Passive RFID <ul><li>Photosensor </li></ul><ul><ul><li>Most compatible with passive batteryless RFID </li></ul></u...
Future Directions <ul><li>Sophisticated tags </li></ul><ul><li>Light modulation </li></ul><ul><li>Handheld projector </li>...
Comparison with Bar-codes <ul><li>RFID </li></ul><ul><ul><li>Barcodes take up more space </li></ul></ul><ul><ul><li>Long b...
Locate Tags Precisely using a  handheld scanner/projector <ul><li>Handheld device </li></ul><ul><li>Eliminates multiple RF...
<ul><li>Photosensor added to RFID tags </li></ul><ul><li>Hybrid communication via RF and via light </li></ul><ul><li>Solve...
2. Select subset of tags for operation <ul><li>Advantages </li></ul><ul><li>Only tags which are illuminated respond to que...
Futures <ul><li>We are used to fixed displays like desktops, laptops, and static projectors. </li></ul><ul><li>Handheld de...
Interactive Projection <ul><li>Desktop like interaction </li></ul><ul><ul><li>Cursor follows use motion </li></ul></ul><ul...
Upcoming SlideShare
Loading in...5
×

Rfig Sig04 Presentation

430

Published on

Published in: Business, News & Politics
0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total Views
430
On Slideshare
0
From Embeds
0
Number of Embeds
1
Actions
Shares
0
Downloads
5
Comments
0
Likes
0
Embeds 0
No embeds

No notes for slide

Rfig Sig04 Presentation

  1. 1. Ramesh Raskar, Paul Beardsley, Jeroen van Baar, Yao Wang, Paul Dietz, Johnny Lee, Darren Leigh, Thomas Willwacher Mitsubishi Electric Research Labs (MERL), Cambridge, MA R F I G Lamps : Interacting with a Self-describing World via Photosensing Wireless Tags and Projectors
  2. 2. Radio Frequency Identification Tags (RFID) microchip Antenna No batteries, Small size, Cost few cents
  3. 3. Warehousing Routing Library Baggage handling Currency Livestock tracking
  4. 4. Conventional Passive RFID
  5. 5. Tagged Books in a Library <ul><li>Id </li></ul><ul><li>List of books in RF range </li></ul><ul><li>No Precise Location Data </li></ul><ul><li>Are books in sorted order ? </li></ul><ul><li>Which book is upside down ? </li></ul>
  6. 6. Where are boxes with Products close to Expiry Date ?
  7. 7. Conventional RF tag Photo-sensing RF tag
  8. 8. Photosensor ? Compatible with RFID size and power needs Projector ? Directional transfer, AR with Image overlay
  9. 9. b . Tags respond with date and precise (x,y) pixel location, Projector beams ‘Ok’ at that location a . Photosensing RFID tags are queried via RF c . Multiple users can simultaneously work from a distance without RF collision
  10. 11. RFID (Radio Frequency Identification) RFI G (Radio Frequency Id and Geometry )
  11. 12. Prototype Tag RF tag + photosensor
  12. 13. Outline <ul><li>Photosensing RF tags </li></ul><ul><ul><li>Location sensing </li></ul></ul><ul><li>Geometric Operations </li></ul><ul><ul><li>Multi-view analysis </li></ul></ul><ul><ul><li>Deformation </li></ul></ul><ul><li>Interactive Projection </li></ul><ul><ul><li>Mouse-like interaction </li></ul></ul><ul><ul><li>Image stabilization </li></ul></ul>
  13. 14. Outline <ul><li>Photosensing RF tags </li></ul><ul><ul><li>Location sensing </li></ul></ul><ul><li>Geometric Operations </li></ul><ul><ul><li>Multi-view analysis </li></ul></ul><ul><ul><li>Deformation </li></ul></ul><ul><li>Interactive Projection </li></ul><ul><ul><li>Mouse-like interaction </li></ul></ul><ul><ul><li>Image stabilization </li></ul></ul><ul><li>Applications </li></ul>
  14. 15. Pattern MSB Pattern MSB-1 Pattern LSB Projected Sequential Frames <ul><li>Handheld Projector beams binary coded stripes </li></ul><ul><li>Tags decode temporal code </li></ul>
  15. 16. Pattern MSB Pattern MSB-1 Pattern LSB Projected Sequential Frames <ul><li>Handheld Projector beams binary coded stripes </li></ul><ul><li>Tags decode temporal code </li></ul>
  16. 17. Pattern MSB Pattern MSB-1 Pattern LSB Projected Sequential Frames <ul><li>Handheld Projector beams binary coded stripes </li></ul><ul><li>Tags decode temporal code </li></ul>
  17. 18. Pattern MSB Pattern MSB-1 Pattern LSB Projected Sequential Frames <ul><li>Handheld Projector beams binary coded stripes </li></ul><ul><li>Tags decode temporal code </li></ul>
  18. 19. Pattern MSB Pattern MSB-1 Pattern LSB Projected Sequential Frames <ul><li>Handheld Projector beams binary coded stripes </li></ul><ul><li>Tags decode temporal code </li></ul>
  19. 20. Pattern MSB Pattern MSB-1 Pattern LSB <ul><li>For each tag </li></ul><ul><li>From light sequence, decode x and y coordinate </li></ul><ul><li>Transmit back to RF reader ( Id , x, y ) </li></ul>0 1 1 0 0 X=12
  20. 21. Visual feedback of 2D position <ul><li>Receive via RF {( x 1 ,y 1 ), ( x 2 ,y 2 ), …} pixels </li></ul><ul><li>Illuminate those positions </li></ul>
  21. 22. Limitations <ul><li>Line of sight </li></ul><ul><ul><li>Surface patches </li></ul></ul><ul><ul><li>Multiple tags per object </li></ul></ul><ul><li>Ambient light </li></ul><ul><ul><li>Modulated infrared </li></ul></ul><ul><li>Surface reflectance and shadows </li></ul>
  22. 23. Outline <ul><li>Photosensing RF tags </li></ul><ul><ul><li>Location sensing </li></ul></ul><ul><li>Geometric Operations </li></ul><ul><ul><li>Multi-view analysis </li></ul></ul><ul><ul><li>Deformation </li></ul></ul><ul><li>Interactive Projection </li></ul><ul><ul><li>Mouse-like interaction </li></ul></ul><ul><ul><li>Image stabilization </li></ul></ul><ul><li>Applications </li></ul>
  23. 24. Find 3D coordinates <ul><li>Observe structured patterns via Camera </li></ul><ul><li>Triangulate cam/proj view </li></ul>
  24. 25. 3D from 2 Projector Views (Structure from Motion) <ul><li>Two+ unknown projector views </li></ul><ul><li>Correspondence is trivial </li></ul><ul><li>Applications </li></ul><ul><ul><li>Detect 3D deformations </li></ul></ul><ul><ul><li>Trajectory grouping </li></ul></ul>
  25. 26. Change Detection without fixed camera, in any lighting condition Compare with new coordinates from a different view Record coordinates of tags from one view Before After
  26. 27. Texture Adaptation
  27. 28. Outline <ul><li>Photosensing RF tags </li></ul><ul><ul><li>Location sensing </li></ul></ul><ul><li>Geometric Operations </li></ul><ul><ul><li>Multi-view analysis </li></ul></ul><ul><ul><li>Deformation </li></ul></ul><ul><li>Interactive Projection </li></ul><ul><ul><li>Desktop-like interaction </li></ul></ul><ul><ul><li>Image stabilization </li></ul></ul><ul><li>Applications </li></ul>
  28. 29. Desktop-like Interaction Selecting tags
  29. 30. Support for handheld projection
  30. 31. Mouse Simulation <ul><li>Cursor follows handheld projector motion </li></ul><ul><li>Pre-warped image remains stable </li></ul>
  31. 32. Image Quasi-Stabilization Eliminate hand jitter using inertial sensors+camera
  32. 33. Absolute Stabilization Image stays registered with world features
  33. 34. Image Stabilization
  34. 35. Interactive Projection (Also in Emerging Technologies Booth)
  35. 36. Adaptive Projection ‘ Paste’ : Geometric and Photometric compensation
  36. 37. Prototype Handheld Projector
  37. 38. Related Work <ul><li>Smart objects </li></ul><ul><ul><li>Smart-its </li></ul></ul><ul><ul><li>FindIT Flashlight (Ma and Paradiso 2002) </li></ul></ul><ul><li>Location sensing </li></ul><ul><ul><li>Multiple readers, large antennas </li></ul></ul><ul><ul><li>Olivetti Active Badge , Xerox PARCtab (Want 1995) </li></ul></ul><ul><li>Interaction and Display </li></ul><ul><ul><li>Augmented Reality </li></ul></ul><ul><ul><li>Projector-based AR, Shaderlamps </li></ul></ul><ul><ul><ul><li>(Raskar 1998, Underkoffler 1999, Pinhanez 2001) </li></ul></ul></ul><ul><ul><li>Handheld projector (Raskar 2003) </li></ul></ul>
  38. 39. Applications <ul><li>Single Tags </li></ul><ul><ul><li>Authoring for AR, Store augmentation: display per object </li></ul></ul><ul><li>Multiple tags on objects </li></ul><ul><ul><li>3D Pose </li></ul></ul><ul><ul><li>Packing and placement strategy </li></ul></ul><ul><ul><li>Robot navigation </li></ul></ul><ul><li>Interaction between tagged objects </li></ul><ul><ul><li>History of geometric interaction </li></ul></ul><ul><ul><li>Orientation mismatch </li></ul></ul><ul><li>Distributed tags </li></ul><ul><ul><li>Deformation </li></ul></ul><ul><ul><li>Interpolated values for temperature sensors </li></ul></ul>
  39. 40. Acknowledgements <ul><li>MERL </li></ul><ul><ul><li>Cliff Forlines </li></ul></ul><ul><ul><li>Joe Marks, </li></ul></ul><ul><ul><li>Dick Waters, Kent Wittenburg </li></ul></ul><ul><ul><li>Vlad Branzoi </li></ul></ul><ul><ul><li>Rebecca Xiong, Debbi van Baar </li></ul></ul><ul><li>Mitsubishi Electric, Japan </li></ul><ul><ul><li>Mamuro Kato, </li></ul></ul><ul><ul><li>Keiichi Shiotani </li></ul></ul>
  40. 41. Interacting with a Self-Describing World <ul><li>Hybrid optical and RF communication </li></ul><ul><ul><li>Photosensing Wireless Tags + Projector illumination </li></ul></ul><ul><li>Geometric Analysis </li></ul><ul><ul><li>Location, Selection, Augmentation At a distance </li></ul></ul><ul><li>Interactive Projection </li></ul><ul><ul><li>Desktop like interaction </li></ul></ul><ul><ul><li>Stabilized images </li></ul></ul><ul><li>RFIG Applications beyond logistics </li></ul><ul><ul><li>Identity, Geometry, History, Annotation </li></ul></ul>www.MERL.com
  41. 42. Find tag location using handheld Projector Photosensing Wireless Tags Many geometric ops R F I R F I D Interactive stabilized projection (Radio Frequency Id & Geometry ) G
  42. 43. (End of Presentation)
  43. 44. Towards Passive RFID <ul><li>Photosensor </li></ul><ul><ul><li>Most compatible with passive batteryless RFID </li></ul></ul><ul><ul><li>Power: near zero power for sensing </li></ul></ul><ul><ul><li>Size: Each photocell is tens of micrometer (smaller than RFID which are about 500 micrometer) </li></ul></ul><ul><ul><li>Very low cost, can be built with silicon of RFID microchip </li></ul></ul><ul><ul><li>RFID+Photosensor can be size of grain of rice </li></ul></ul><ul><ul><li>RFIG tag is ‘visible’ to projector but not to humans </li></ul></ul><ul><ul><li>Current prototype uses battery because passive tags are difficult to program </li></ul></ul><ul><li>Other possibilities </li></ul><ul><ul><li>Adding a light emitting diode (LED) </li></ul></ul><ul><ul><ul><li>Requires on board power </li></ul></ul></ul><ul><ul><ul><li>Size has to be considerably large </li></ul></ul></ul><ul><ul><ul><li>Size and power requirements are high </li></ul></ul></ul>
  44. 45. Future Directions <ul><li>Sophisticated tags </li></ul><ul><li>Light modulation </li></ul><ul><li>Handheld projector </li></ul>
  45. 46. Comparison with Bar-codes <ul><li>RFID </li></ul><ul><ul><li>Barcodes take up more space </li></ul></ul><ul><ul><li>Long bit sequences </li></ul></ul><ul><ul><li>Multiple tags can be simultaneously read </li></ul></ul><ul><ul><li>Read distance is high </li></ul></ul><ul><li>RFIG </li></ul><ul><ul><li>Multiple barcodes per object is cumbersome </li></ul></ul><ul><ul><li>Difficult to attach sensor (e.g. temperature) </li></ul></ul>
  46. 47. Locate Tags Precisely using a handheld scanner/projector <ul><li>Handheld device </li></ul><ul><li>Eliminates multiple RF readers for triangulation or large antennas </li></ul><ul><li>Projector indicates location of tag </li></ul>Illuminate tags with binary coded stripes with projector
  47. 48. <ul><li>Photosensor added to RFID tags </li></ul><ul><li>Hybrid communication via RF and via light </li></ul><ul><li>Solve 3 main issues with conventional RFIDs </li></ul><ul><li>Locate tags precisely </li></ul><ul><li>Select a subset of tags for operation </li></ul><ul><li>Long distance operation without collision </li></ul><ul><li>Applications beyond inventory management </li></ul><ul><li>Warehouse: locate objects for specific queries </li></ul><ul><li>Training and maintenance: interactive augmented reality </li></ul><ul><li>Surveillance: keep trail of object locations </li></ul>
  48. 49. 2. Select subset of tags for operation <ul><li>Advantages </li></ul><ul><li>Only tags which are illuminated respond to queries </li></ul><ul><li>No RF collision between multiple tags </li></ul><ul><li>Projector also use as a display to indicate result of operation </li></ul><ul><li>Technique </li></ul><ul><li>Illuminate subset of tags with flashlight or projector </li></ul>3. Long distance operation without RF collision
  49. 50. Futures <ul><li>We are used to fixed displays like desktops, laptops, and static projectors. </li></ul><ul><li>Handheld devices with screens have become common over the last decade – but the displays are small. </li></ul><ul><li>Handheld projectors offer a new display modality with its own characteristics - portable device size but with a large display. </li></ul><ul><li>Add interaction to achieve ‘the world as a desktop’… </li></ul>
  50. 51. Interactive Projection <ul><li>Desktop like interaction </li></ul><ul><ul><li>Cursor follows use motion </li></ul></ul><ul><ul><li>Center pixel is the pointer </li></ul></ul><ul><ul><li>Rest of the image is stabilized </li></ul></ul><ul><ul><li>Selection </li></ul></ul><ul><li>Image stabilization </li></ul><ul><ul><li>Absolute: camera and inertial sensor </li></ul></ul><ul><ul><li>Relative: world markers </li></ul></ul><ul><li>Copy-paste </li></ul><ul><ul><li>Camera captures texture during ‘copy’ </li></ul></ul><ul><ul><li>Projection after geometric/photometric adaptation </li></ul></ul>
  1. A particular slide catching your eye?

    Clipping is a handy way to collect important slides you want to go back to later.

×