Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Presentatie Van Mm X(Final2)

  • Login to see the comments

  • Be the first to like this

Presentatie Van Mm X(Final2)

  1. 1. Willem Mestdagh - Kevin Cordemans
  2. 2. Papers <ul><ul><ul><li>MS Research </li></ul></ul></ul><ul><ul><ul><li>PlayAnywhere: Compact Interactive Tabletop Projection-Vision System [] </li></ul></ul></ul><ul><ul><ul><li>Andy Wilson </li></ul></ul></ul><ul><ul><ul><li>Thinsight: Integrated Optical Multi-touch Sensing through Thin Form-factor Displays [] </li></ul></ul></ul><ul><ul><ul><li>Bill Buxton, Steve Hodges, ... </li></ul></ul></ul>
  3. 3. PlayAnywhere <ul><li>PlayAnywhere: A Compact Interactive Tabletop Projection-Vision System </li></ul><ul><li>Andy Wilson at Microsoft Research, Redmond </li></ul><ul><li>2005 </li></ul>
  4. 4. Introduction Looks great... But... Calibration? Installation? Portability? Cost?
  5. 5. Mainstream Consumer Acceptance <ul><li>Multi-touch: new sensing and display technologies </li></ul><ul><li>Ubiquitious computing </li></ul><ul><li> </li></ul><ul><li>Portability </li></ul><ul><li>Ease of installation </li></ul><ul><li>Ease of use: any surface? </li></ul><ul><li>Calibration </li></ul><ul><li>Cost </li></ul><ul><li> Devise an easy-to-use, low-cost multi-touch system... </li></ul>
  6. 6. PlayAnywhere <ul><li>Front-projected computer vision-based interactive table system which uses a new projecting technology to obtain an exceptionnaly self-contained form factor </li></ul><ul><li>Requirements: </li></ul><ul><ul><li>Quick set up: no calibration beyond factory </li></ul></ul><ul><ul><li>No “installation” </li></ul></ul><ul><ul><li>Usable on any flat surface </li></ul></ul><ul><ul><li>Compact but still displaying and sensing over a large surface (form factor) </li></ul></ul><ul><ul><li>General purpose </li></ul></ul><ul><ul><li>(Low cost) </li></ul></ul>
  7. 7. Future? <ul><li>Canesta projection keyboard </li></ul>
  8. 8. PlayAnywhere <ul><li>What is it? </li></ul><ul><li>Known problems in existing systems </li></ul><ul><li>How does it work? </li></ul><ul><ul><li>PlayAnywhere configuration </li></ul></ul><ul><ul><ul><li>Projector </li></ul></ul></ul><ul><ul><ul><li>Camera and IR Illuminant </li></ul></ul></ul><ul><ul><li>Image Processing </li></ul></ul><ul><ul><ul><li>Image Rectification </li></ul></ul></ul><ul><ul><ul><li>Touch and Hover </li></ul></ul></ul><ul><ul><ul><li>Analysis of shadow </li></ul></ul></ul><ul><ul><ul><li>PlayAnywhere Visual Code </li></ul></ul></ul><ul><ul><ul><li>Flow Move </li></ul></ul></ul><ul><ul><li>Computer Vision Pros and Cons </li></ul></ul>
  9. 9. Known problems in existing systems
  10. 10. PlayAnywhere <ul><li>What is it? </li></ul><ul><li>Known problems in existing systems </li></ul><ul><li>How does it work? </li></ul><ul><ul><li>PlayAnywhere configuration </li></ul></ul><ul><ul><ul><li>Projector </li></ul></ul></ul><ul><ul><ul><li>Camera and IR Illuminant </li></ul></ul></ul><ul><ul><li>Image Processing </li></ul></ul><ul><ul><ul><li>Image Rectification </li></ul></ul></ul><ul><ul><ul><li>Touch and Hover </li></ul></ul></ul><ul><ul><ul><li>Analysis of shadow </li></ul></ul></ul><ul><ul><ul><li>PlayAnywhere Visual Code </li></ul></ul></ul><ul><ul><ul><li>Flow Move </li></ul></ul></ul><ul><ul><li>Computer Vision Pros and Cons </li></ul></ul>
  11. 11. PlayAnywhere Configuration All-in-One IR illuminant Projector (with mirrors) Camera with IR-passing-filter Prototype
  12. 12. Projector <ul><li>NEC WT600 DLP </li></ul><ul><ul><li>40’’ diagonale afbeelding </li></ul></ul><ul><ul><li>Voordelen van de plaats van de projector </li></ul></ul><ul><ul><ul><li>Gevaarlijke plaatsing projector is vermeden </li></ul></ul></ul><ul><ul><ul><li>Wanneer camera en projector samen zijn geplaatst </li></ul></ul></ul><ul><ul><ul><li> geen nood om de camera te hercalibreren bij verplaatsing van het toestel </li></ul></ul></ul><ul><ul><li>Nadelen </li></ul></ul><ul><ul><ul><li>Beeldkwaliteit kan niet gegarandeerd worden, want projectie-oppervlak is niet gekend </li></ul></ul></ul>
  13. 13. Camera en IR Illuminant <ul><li>De projector belicht de scene met zichtbaar straling </li></ul><ul><li>De illuminant belicht de scene met infrarode straling </li></ul><ul><li>Camera ontvangt alleen gereflecteerde infrarode stralen dankij de infrared-passing-filter die aan de camera bevestigd is </li></ul><ul><li>(Camera blind voor reflecties van de projectorstraling) </li></ul>
  14. 14. How does it work? <ul><li>Image Processing </li></ul><ul><ul><li>Preparation: Image Rectification </li></ul></ul><ul><ul><li>Computer Vision Techniques: Analysis of Shadow </li></ul></ul><ul><ul><ul><li>Hover and Touch </li></ul></ul></ul><ul><ul><ul><li>Flow Move </li></ul></ul></ul><ul><ul><ul><li>PlayAnywhere Visual Code </li></ul></ul></ul>
  15. 15. Image Rectification (1)  Remove distortion via standard bilinear interpolation techniques <ul><li>Parameters for transformations need only be determined once (factory): </li></ul><ul><li>Lens specs known </li></ul><ul><li>Fixed topology device-plane (assumption) </li></ul>Setup Good thing Bad thing Wide lens Maximize usable surface Creates barrel distortion Oblique position of camera Optimizes form factor (compact device) Produces projective distortion
  16. 16. Image Rectification (2)
  17. 17. Image Rectification (3) <ul><li>Input image and projected image are in a one-to-one correspondence, meaning: </li></ul><ul><ul><li>A rectangular object on the surface appears as a rectangular object in the input image at the same (scaled) coordinates. </li></ul></ul><ul><li>Rectified image suited for computer vision techniques </li></ul><ul><li>Limitation: objects further away from the unit will appear at a lower resolution (Minimum effective resolution is less than that of the acquired image) </li></ul>
  18. 18. Hover and Touch <ul><li>Detect touch without relying on special instrumentation of the surface, so that the device may operate on any available flat surface (goal) </li></ul><ul><li>PlayAnywhere: exploite change in appearance of shadows as an object approaches the surface </li></ul><ul><li>No tracking of fingers, ... </li></ul>
  19. 19. Hover and Touch <ul><li>thresholding operation </li></ul>Hover Candidate finger positions: highest on each of the distinct shadows which enter the scene from the bottom of the image Touch Analysis of shadow shapes: threshold width of finger shadow at a location slightly below the topmost point
  20. 20. Hover and Touch <ul><li>To limit false positives, candidate finger must lie on a shadow that extends to the bottom of the image </li></ul><ul><ul><ul><li> Objects on table can corrupt touch detection by mimicing arm shadows, meaning their shadows extends to the bottom of the image </li></ul></ul></ul><ul><li>Presently, only one finger per hand (more fingers requires more spohisticated analysis) </li></ul><ul><li>Precision limited by image resolution </li></ul>
  21. 21. Flow Move <ul><li>Support direct manipulation of virtual objects (doc’s, photo’s) </li></ul><ul><li>Gestures that are natural to a specific activity </li></ul><ul><ul><li>E.g. resizing and rotating photo’s as if they were lying on a table </li></ul></ul><ul><li>PlayAnywhere: Optical flow techniques </li></ul><ul><ul><li>Based on local texture on the movig object </li></ul></ul><ul><ul><li>No absolute position-based tracker, no widgets </li></ul></ul><ul><ul><li>Instead simple statistics summarizing the flow field, computing only relative motion information </li></ul></ul>
  22. 22. Optical Flow (1) <ul><li>Optical Flow computed from the most recently acquired rectified image I(t) and the previous image I(t-1) </li></ul><ul><li>Mask images with grid </li></ul>Image (t-1) Image (t)
  23. 23. Optical Flow (2) <ul><ul><li>For each (x,y) on a regular grid, the integer vector quantity (dx, dy) is determined, such that: </li></ul></ul><ul><ul><li>image patch P centered on (x, y) at time t-1 most closely matches </li></ul></ul><ul><ul><li>image patch P’ centered on (x+dx, y+dy) at time t. </li></ul></ul><ul><ul><li>Image patches are compared by computing pixelwise absolute differences (low values  close match). </li></ul></ul><ul><ul><li>If image patch P closely matches image patch P’, there is a good chance that the two patches are representing the same content, shifted to a different gridpoint in the interval. </li></ul></ul><ul><ul><li>(dx, dy) gives the direction of this shift. </li></ul></ul>Image(t) (x+dx, y+dy) Image patch P’ Image(t-1) (x, y) Image patch P Resulting vector
  24. 24. Optical Flow (3) For a given patch in the image, select (dx, dy) that minimizes Vector field can be used as an argument in a function to infer meaning from user’s gesture Resulting vector field
  25. 25. PlayAnywhere Visual Codes <ul><li>Ook mogelijk te werken met infrarood systemen in objecten  mogelijkheid tot werken met barcodes (derde vb met barcode) </li></ul>
  26. 26. Resultaat <ul><ul><ul><ul><li> </li></ul></ul></ul></ul>Nu : Thingsight
  27. 27. Beschrijving <ul><li>“ Optical sensing system, fully integrated into a thin form factor display” </li></ul><ul><li>Scherm met multi-touch systeem </li></ul><ul><li>2 mogelijkheden om multitouchsystemen uit te werken : </li></ul><ul><ul><li>- een arrangement van electroden op het oppervlak </li></ul></ul><ul><ul><li>- werken met cameras : voor of achter het scherm </li></ul></ul>
  28. 28. Vergelijking <ul><li>Beide Touch-algoritme op basis van schaduwinval van handen </li></ul><ul><li>Beide Handig te gebruiken en te verplaatsen </li></ul><ul><li>Principe werking ongeveer gelijk </li></ul><ul><ul><li> verschil : thinsight werkt met elektroden op het scherm, playanywhere werkt met camera’s boven het scherm </li></ul></ul><ul><li>Beiden hebben de mogelijkheid tot werken met externe objecten met een infrarode bron </li></ul><ul><li>Voordeel bij thinsight : geen oppervlakte voor apparaat nodig </li></ul>