Downloaded 16 times
Uploaded byClemente Giorio
Dev09 – la battaglia del touchless
The document outlines a presentation on natural user interfaces and various technologies like Microsoft Kinect, Intel RealSense, and Leap Motion, focusing on their capabilities and applications. It compares features, specifications, and use cases for each technology, highlighting pros and cons for developers. The document emphasizes the importance of user posture and environment in designing effective interactions with these systems.
More Related Content
Similar to Dev09 – la battaglia del touchless
Dev09 – la battaglia del touchless
- 1. DEV09 – LABATTAGLIA DEL TOUCHLESS Matteo Valoriani @MatteoValoriani Clemente Giorio @Tinux80
- 2. #CDays15 – Milano24, 25 e 26 Marzo 2015 Grazie a Platinum Sponsor
- 3. #CDays14 – Milano25, 26 e 27 Febbraio 2014 Who we are… Matteo Valoriani Microsoft MVP Kinect for Windows Intel Software Innovator PhD on HCI @Politecnico di Milano CEO of Fifth Ingenium Consultant Clemente Giorio Microsoft MVP Kinect for Windows Senior Developer at Crionet srl Instructor at Overnet Author at Packt Publishing Consultant
- 4. #CDays14 – Milano25, 26 e 27 Febbraio 2014 Agenda • HCI evolution • Microsoft Kinect • Intel Real Sense • Leap Motion • Duo3D • Myo • MaindWave
- 5. UI evolution 5Natural UserInterface
- 6. #CDays15 – Milano24, 25 e 26 Marzo 2015 You have to be a magician… or at least a good illusionist
- 7. #CDays14 – Milano25, 26 e 27 Febbraio 2014 http://youtu.be/Hi5kMNfgDS4
- 8. #CDays15 – Milano24, 25 e 26 Marzo 2015 Kinect 2 - Specs 3D DEPTH SENSOR RGB CAMERA MULTI-ARRAY MIC Hardware: Depth resolution: 512×424 RGB resolution: 1920×1080 (16:9) FrameRate: 30 FPS Latency: 60 ms
- 9. #CDays15 – Milano24, 25 e 26 Marzo 2015
- 10. #CDays15 – Milano24, 25 e 26 Marzo 2015 Kinect 2 vs Kinect 1 Version 1 Version 2 Depth range 0.4m → 4.0m 0.4m → 4.5m Color stream 640×480 (VGA) 1920×1080 (Full HD) Depth stream 320×240 @8bits 512×424 @16bits Infrared stream None 512×424 Audio stream 4-mic array 16 kHz 4-mic array 48 kHz USB 2.0 3.0 Hand Traking External tools Yes Face Traking Yes Yes+Expressions FOV 57° H 43° V 70° H 60° V Tilt Motorized Manual
- 11. #CDays14 – Milano25, 26 e 27 Febbraio 2014 demo
- 12. Cons: The device islarge A dedicated power cord is required The amount of data generated by the sensor also tends to saturate a USB controller Kinect cannot easily distinguish individual fingers on a hand Most of the features listed above require the Microsoft SDK, which is only supported for desktop applications on Windows 7 and 8. Pros Vs. Cons Pros: Skeletal tracking Face tracking Multiple sensors: Raw data Voice control Best for: Kiosks, installations, and digital signage projects where the user will be standing fairly far away from the display.
- 13. #CDays14 – Milano25, 26 e 27 Febbraio 2014
- 14. #CDays14 – Milano25, 26 e 27 Febbraio 2014
- 15. Understands 4 basictypes of input 15Intel® RealSense™ Hands-On Lab - Milan Categories of Input Capabilities Features Hands • Hand and Finger Tracking • Gesture Recognition • 22-point Hand and Finger Tracking • 9 static and dynamic mid-air gestures Face • Face Detection and Tracking • Multiple Face Detection and tracking • 78-point Landmark Detection (facial features) • Emotion Recognition (7 emotions, coming post-Beta) • Pulse Estimation • Face Recognition (Coming post-beta) Speech • Speech Recognition • Command and Control • Dictation • Text to Speech Environment • Segmentation • 3D Scanning • Augmented Reality • Background Removal • 3D Object / Face / Room Scanning (Coming post-beta) • 2D/3D Object Tracking • Scene Perception (coming post-beta)
- 16. Understands Hardware Limits 16Intel®RealSense™ Hands-On Lab - Milan
- 17. #CDays14 – Milano25, 26 e 27 Febbraio 2014 demo
- 18. Cons: Getting some ofthe deeper features (like age and gender detection) to work is a bit tricky. Device and software are in beta. Due to the close range of the tracking system, hand gestures must be designed such that a user’s hand doesn’t occlude their own view of the display. Pros Vs. Cons Pros: Smaller and less expensive Close-range tracking Hand posture/gesture recognition Facial analysis Speech (built-in support for speech synthesis powered by Nuance). Raw data Framework support: Processing, Unity and OpenFrameworks Best for: Desktop/laptop applications where the user will be seated in front of the PC. Close range applications where features, apart from hand tracking and recognition, are necessary without too much precision or accuracy.
- 19. #CDays14 – Milano25, 26 e 27 Febbraio 2014 https://www.youtube.com/watch?v=_d6KuiuteIA https://airspace.leapmotion.com/
- 20.
- 21. Leap Motion -Fieldof View 150° - Long Side 120° - Short Side Max 60 cm above the controller Max 60 cm wide on each side
- 22. Leap Motion forDevelopers
- 23. #CDays14 – Milano25, 26 e 27 Febbraio 2014 demo
- 24. Cons: Sensing range isfairly limited. Only fingers are tracked. There is no skeleton or face tracking. No access to the raw sensor data. Pros Vs. Cons Pros: Finger tracking is fast and accurate. HMD Mountable Smaller and less expensive Framework support: .NET, Processing, Cinder, etc. Compatible: Mac OS and Windows Best for: Controlled kiosk environments with a pointing-based UI. Generally best for general audience desktop apps which can be distributed in the Airspace store. Head-mounted display video pass-through. Augmented Reality. Computer Vision.
- 25. #CDays14 – Milano25, 26 e 27 Febbraio 2014 https://youtu.be/XNztZ2Chmzg
- 26. Cons: SDK Samples Device API Dense3DAPI Documentation Pros Vs. Cons Pros: Ultra Compact Design (52x25x11mm). Fast (360 FPS @320x120, 56FPS @752x480). FoV 170° with Low Distortion ( < 3% ) Acceleromet, gyroscope and temperature sensor. HMD Mountable Outdoor with environmental IR. Cross Platform Best for: Augmented Reality and Virtual Reality HeadSet. Outdoor, portable applications. Mird-range depth sensing with low latency.
- 27. Leap, RealSense, Kinect 27Intel®RealSense™ Hands-On Lab - Milan 2,5 cm 60 cm 2 m 4 m
- 28. #CDays14 – Milano25, 26 e 27 Febbraio 2014
- 29. #CDays14 – Milano25, 26 e 27 Febbraio 2014 demo
- 30. Cons: Calibration Dry Eyes Near range ProsVs. Cons Pros: Accuracy Low Latency Lenses and glasses support Price Best for: EyeTraking, Marketing Analysis, Medical device option
- 31. #CDays14 – Milano25, 26 e 27 Febbraio 2014
- 32. #CDays14 – Milano25, 26 e 27 Febbraio 2014 Hand Gestures
- 33. #CDays14 – Milano25, 26 e 27 Febbraio 2014 demo
- 34. Cons: Confort One devicefor each user. For complex scenarios required multiple devices Pros Vs. Cons Pros: Wearable Hand gestures Multi Platform (Windows, Mac, iOS and Android) Unity support Best for: Outdoor, portable applications.
- 37. #CDays14 – Milano25, 26 e 27 Febbraio 2014 demo
- 38. Cons: Confort One devicefor each user Few sensors Pros Vs. Cons Pros: Wearable Cheap Hand gestures Multi Platform (Windows, Mac, iOS and Android) Best for: Outdoor, portable applications.
- 39. #CDays14 – Milano25, 26 e 27 Febbraio 2014 Recap
- 40. Some selection criterias… 1°2° 3° 4° 2 or + users Kinect 2 Intel Leap Full body interaction Kinect 2 - - Hand Gesture Recognition Myo Intel Leap Kinect 2 Accuracy Leap Intel Kinect 2 Myo Voice command Intel Kinect 2 FaceTraking Tobii Intel Kinect Commecial use Kinect 2 Intel Tobii Leap Myo Compatibility Leap MYO Tobii Intel Kinect 2
- 41. Capture Volumes 41Intel® RealSense™Hands-On Lab - Milan The user is performing a hand gesture outside of the capture volume. The camera will not see this gesture
- 42. Evaluate different settingsand environment 42Intel® RealSense™ Hands-On Lab - Milan
- 43. Sensor with Camerause IR light and Sunlight can blind the camera!!! • Check exposition during all day • Verify that there isn’t direct light on the camera Indoor/Outdoor 43Intel® RealSense™ Hands-On Lab - Milan
- 44. These devices aren’ta Rugged devices: • Check temperatures (+3/33°) • Check humidity Indoor/Outdoor (2) 44Intel® RealSense™ Hands-On Lab - Milan
- 45. Comfortable positions Your usersare not GORILLAS!!! Intel® RealSense™ Hands-On Lab - Milan 45
- 46. User posture mayaffect design of a gesture Intel® RealSense™ Hands-On Lab - Milan 46
- 47. Input variability Intel® RealSense™Hands-On Lab - Milan 47
- 48. Feedback, feedback, feedback,… 48Intel®RealSense™ Hands-On Lab - Milan View of user: • User Viewport • User Overlay
- 49. … where actionsperformed for some other purpose or unconscious signs are interpreted in order to influence/improve/facilitate the actors' future interaction or day-to-day life (from Alan Dix) • The interaction is not purposeful from the person side, but it is designed “to happen” • It “happens” in relation to signs which are not done for that (body temperature, unconscious reactions such as blink rate, or unconscious aspects of activities such as typing rate, vocabulary shifts (e.g. modal verbs), actions done for other purposes, … • It is designed for people acting Manage Incidental Interaction 49Intel® RealSense™ Hands-On Lab - Milan
- 50. #CDays14 – Milano25, 26 e 27 Febbraio 2014 Q&A Tutto il materiale di questa sessione su http://www.communitydays.it/ Lascia subito il feedback su questa sessione, potrai essere estratto per i nostri premi! Seguici su Twitter @CommunityDaysIT Facebook http://facebook.com/cdaysit #CDays15
Editor's Notes
- #3 Slide da mostrare prima di iniziare la sessione – non rimuovere!
- #11 Kinect for Windows v2: - Up to 6 skeletons with 25 joints (each) - Allows tracking IR reflective objects - ToF - Only USB 3.0 - Doesn’t support most ov USB over Ethernet extenders. - Only Windows 8.x - Current SDK support only one Kinect per PC - 115W Kinect for Windows v1: - Up to 2 skeletons with 20 joints - Structured Light (near-IR light pattern) - raw depth maps from the sensor show an observable quantization effect. - Windows 7/8 - Multiple Kinect - 12.96W
- #17 Intel RealSense for tablet: 2.5W 628x468 Depth Resolution Indoor - Projects an IR pattern to the environment and uses stereo matching to obtain the depth map. Outdoor - Stereo matching without IR pattern
- #21 https://developer.leapmotion.com/documentation/csharp/api/Leap.Gesture.html http://pluralsight.com/training/Courses/TableOfContents/developing-leap-motion-csharp Each cameras are 640x240
- #24 https://www.youtube.com/watch?v=gby6hGZb3ww&t=27
- #26 The DUO mini lx is an ultra-compact imaging sensor with a standard USB interface. Intended for use in research, industrial and integration, the camera's high speed and small size make it ideal for existing and new use cases for vision based applications. With a programmable illumination board and built-in IR filters it allows for precise control of lighting environment. Delivering configurable and precise stereo imaging for robotics, inspection, microscopy, human computer interaction and beyond. The solution consists of: Stereo Imaging with Global Shutter Hi-Speed USB 2.0 Interface Six Degree of Freedom (DoF) Accelerometer/Gyroscope IMU/Temperature Fully Programmable LED Array (3xIR High Power LEDs) DUO SDK License USB Mini-B Cable Offering a board level solution for 3D sensing utilizing stereo vision, ready to work out of the box supporting a wide range of accessories and configurations. Hardware Ultra Compact Design 30mm Baseline Configurable Framerates RAW Sensor Data 170° Wide Angle M8 Lenses 52x25x11mm Software DUO Device API/SDK Linux/OSX/Windows Drivers Standard C API with Bindings Low Level Access & Control Samples & Solutions
- #27 Illumination Fully Programmable LED Array. 3 Independently controlled 3.4W 850nm IR LEDs 170° light cone in 256 linear steps. Lens Mount Standards M8 x P0.5 Filters 850-870nm Narrow Band-Pass Power Consumption 2.5W Weight 12.5g
- #29 The DUO mini lx is an ultra-compact imaging sensor with a standard USB interface. Intended for use in research, industrial and integration, the camera's high speed and small size make it ideal for existing and new use cases for vision based applications. With a programmable illumination board and built-in IR filters it allows for precise control of lighting environment. Delivering configurable and precise stereo imaging for robotics, inspection, microscopy, human computer interaction and beyond. The solution consists of: Stereo Imaging with Global Shutter Hi-Speed USB 2.0 Interface Six Degree of Freedom (DoF) Accelerometer/Gyroscope IMU/Temperature Fully Programmable LED Array (3xIR High Power LEDs) DUO SDK License USB Mini-B Cable Offering a board level solution for 3D sensing utilizing stereo vision, ready to work out of the box supporting a wide range of accessories and configurations. Hardware Ultra Compact Design 30mm Baseline Configurable Framerates RAW Sensor Data 170° Wide Angle M8 Lenses 52x25x11mm Software DUO Device API/SDK Linux/OSX/Windows Drivers Standard C API with Bindings Low Level Access & Control Samples & Solutions
- #30 https://www.youtube.com/watch?v=gby6hGZb3ww&t=27
- #31 Illumination Fully Programmable LED Array. 3 Independently controlled 3.4W 850nm IR LEDs 170° light cone in 256 linear steps. Lens Mount Standards M8 x P0.5 Filters 850-870nm Narrow Band-Pass Power Consumption 2.5W Weight 12.5g
- #32 93g
- #33 Nine-axis IMU tracks motions and rotations (three-axis gyroscope, three-axis accelerometer and three-axis magnetometer) Medical Grade Stainless Steel EMG sensors “catch” muscle activity and allow to detect hand gestures
- #47 When you design a gesture consider different posture of user. Up, sitting, lying
- #48 Some other challenges are: The physical variability of people (tall and low people) Input variability: different people perform same gesture in different ways or use different gesture for the same action depending on cultural origins Test all gestures with the final users!!
- #51 Ultima slide, obbligatoria