Multitouch surface with tangible objects and augmented reality support


Published on

Workshop Winds/UFSCar July,2010
Master Degree from Maximiliano Marques

Published in: Technology
  • Be the first to comment

  • Be the first to like this

No Downloads
Total views
On SlideShare
From Embeds
Number of Embeds
Embeds 0
No embeds

No notes for slide
  • Compartilhado: Mais de um usuário nomeado ou não com diferentes papeisTangível: Manipulação de objetos é intuitívo, ágil e precisoMultimídia: manipulação, compartilhamentoGestual: Facilidade, AgilidadeRA: a) Uso de contextos b) Permite a manipulação do digital sem a barreira do realMóvel: Pode ser movido com certa facilidade, setup tem que ser rápido, corpo único
  • XUL: MozilaXAML: MicrosoftMXML: Adobe
  • Multitouch surface with tangible objects and augmented reality support

    1. 1. Project - Multi-touch surface with tangible objects and augmented reality support <br />Student:MaximilianoSelmi Marques<br />WINDIS Lab( former LRVNet Group ) - UFSCar<br />July/ 2010<br />
    2. 2. Objectives<br />Hardware and Software Prototype Development<br />Table-Based Interface<br />Multimedia<br />Collaborative<br />Multiuser<br />Tangible<br />Augmented Reality Ready<br />Case study: Emergency Management and Unmanned Aerial Vehicle (UAV)<br />Other applications: entertainment, games, monitoring systems, control panel<br />March 2010<br />Maximiliano Selmi Marques <br />2<br />
    3. 3. State of the Art<br />The ReacTable( MTG @ UPF – Barcelona, 2005 )<br />Tangible Electro-Acoustic Music Instrument<br />Multi-touch & Multiuser (Collaborative)<br />Computer Vision Based Solution<br />March 2010<br />Maximiliano Selmi Marques <br />3<br />
    4. 4. Surface Architeture<br />Requisites<br />Multiuser<br />Intuitive<br />Agile<br />Precise<br />Multimedia<br />Mobile (somewhat) <br />Paradigms<br />Shareable<br />Tangible<br />Multimedia<br />Gestural<br />Augmented Reality<br />Mobile<br />March 2010<br />Maximiliano Selmi Marques <br />4<br />
    5. 5. Hardware Prototype<br /><ul><li>Surface
    6. 6. Plastic Film for protection
    7. 7. Paper Film for projection and light diffuser surface
    8. 8. Silicone Rubber for total internal reflection
    9. 9. Plexiglas with IR led frame
    10. 10. Video Camera
    11. 11. With IR Bandpass Filter
    12. 12. Without IR Cut-off Filter
    13. 13. Rear Video Projector
    14. 14. IR Spotlight for pattern illumination</li></ul>March 2010<br />Maximiliano Selmi Marques <br />5<br />
    15. 15. Hardware Prototype<br />March 2010<br />Maximiliano Selmi Marques <br />6<br />
    16. 16. Software Prototype<br />Computer Vision System<br />Touch pattern detection<br />Pattern detection for tangible objects with orientation and position<br />Calibration between camera and projector <br />Background and noise subtraction <br />Multimedia & Graphic User Interface (GUI)<br />Gesture Capture<br />Use of augmented reality to surpass tangible interface common limitations<br />Expansible and modular architecture<br />Open source libraries and toolkits to pattern detection and tracking<br />TUIO Standard approach<br />Protocol<br />API<br />Extended XML Like Standard to describe visual and actions components (ie. XUL, XAML and MXML) <br />March 2010<br />Maximiliano Selmi Marques <br />7<br />
    17. 17. Application Features<br />Security and permissions polices<br />Handle of external events<br />Dispatch user actions<br />Prioritize external events and user actions<br />Resources loading and management<br />Network connection management<br />March 2010<br />Maximiliano Selmi Marques <br />8<br />
    18. 18. Framework Features<br />Tangible object registry<br />Handle internal events<br />Tracking user touch and tangible objects<br />Controller Definition<br />Graphic Component Definition<br />Let application easily assign controllers to graphics objects<br />Bind user and action (if possible)<br />User workspace delimitation <br />Visual Layers implementation<br />Replace real tangible objects to AR Virtual Objects<br />Render AR point of view and layers per user<br />Handle of multimedia content<br />Link with another device to expand surface area<br />March 2010<br />Maximiliano Selmi Marques <br />9<br />
    19. 19. March 2010<br />Maximiliano Selmi Marques <br />10<br />System Architecture<br />
    20. 20. Case Study– Mobile Map<br />Collaborative Control Panel<br />Control of Unmanned Aerial Vehicle<br />Send commands to UAVs<br />Recreate cockpit information<br />Show acquired data<br />Draw routes, position and estimated time of arrival (ETA)<br />Check user permission and polices<br />March 2010<br />Maximiliano Selmi Marques <br />11<br />
    21. 21. Reduced ScopeWhat is IN<br />Single Map<br />One large image with geo reference <br />Waypoints<br />Creation; Edition; Deletion<br />Position & altitude <br />Path<br />Fixed to two paths<br />One objective: Go from first Waypoint to last one<br />Audio Feedback<br />Simple sounds<br />UAV <br />simple movement to follow the path<br />Fixed to two UAV<br />Minimum HUD Info (altitude, speed, heading) <br />AR with Wiimote & iPhone<br />WLAN & Bluetooth<br />iPhone render only<br />Few Tangible Tokens<br />PAN & Zoon<br />Waypoint manipulation (x2)<br />UAV command (with menu) <br />March 2010<br />Maximiliano Selmi Marques <br />12<br />
    22. 22. Reduced ScopeWhat is Out<br />Online Map Integration<br />Server Simulation<br />Unlimited paths<br />GUI manager<br />Security<br />User control<br />Gesture recognition<br />March 2010<br />Maximiliano Selmi Marques <br />13<br />