Intel RealSense Hands-on Lab - Rome Tips and Tricks from Real Case Studies Arguments: - Differences among RealSense Cameras - Limits and constrains of F200 camera - Tips & Tricks to improve user experience - Possible scenarios - Fast and Simplified way to create RealSense applications: NetSense

1. Tips and Tricks from
Real Case Studies
Matteo Valoriani - Roma – 23/09/2015
mvaloriani@gmail.com @matteovaloriani

2. 2
Nice to Meet You
Matteo Valoriani
CEO of FifthIngenium
PhD at Politecnico of Milano
Speaker and Consultant
Microsoft MVP
Intel Software Innovator
email: mvaloriani@gmail.com
twitter: @MatteoValoriani
linkedin: https://it.linkedin.com/in/matteovaloriani

3. 3
You have to be a
magician…
or at least a good
illusionist

4. UI evolution
4

5. RealSense SnapShot

6. Display: 8.4" OLED infinity (2560 x 1600)
Dimensions: 215.9 x 124.2 x 6.1 mm
Weight: 305 g
OS: Android OS, v4.4.2 (KitKat),
CPU: Intel Atom Z3580, Quad-core 2.3 GHz
Memory: 16 GB (+microSD), 2 GB RAM
Primary: 8 MP, 3264 x 2448 pixels,
autofocus
Secondary: 2 MP
Intel RealSense Snapshot, 2 x 720p
cameras
Dell Venue 8 7000
6

7. Measurement Tool
7

8. Refocus / PhotoEditing
8

9. Stereoscopic Vision
9

10. Depth Sensing
10

11. Mathematical Model
b
𝑑 = 𝑥𝑙 − 𝑥 𝑟
𝑏+𝑥 𝑙 − 𝑥 𝑟
𝑍−𝑓
=
𝑏
𝑍
Z =
𝑏∗𝑓
𝑑
11

12. RealSense R200

13. R200
@60FPS, depth at 320x240,, color can be 640x480
@60FPS, depth at 480x360, color can be 320x240 or 640x480
@30FPS, depth at 320x240,, color can be 640x480, 1280x720 or 1920x1080
@30FPS, depth at 480x360, color can be 320x240, 640x480, 1280x720, or 1920x1080
The dual depth cams use a fixed focus 4:3 aspect ratio with a 70x59x46 degree field of view.
The IR is a class 1 laser in the 850nm range

14. RealSense F200

15. Understands 4 basic types of input
15
Categories of
Input
Capabilities Features
Hands • Hand and Finger
Tracking
• Gesture Recognition
• 22-point Hand and Finger Tracking
• 9 static and dynamic mid-air gestures
Face • Face Detection and
Tracking
• Multiple Face Detection and tracking
• 78-point Landmark Detection (facial features)
• Emotion Recognition (7 emotions, coming post-Beta)
• Pulse Estimation
• Face Recognition (Coming post-beta)
Speech • Speech Recognition • Command and Control
• Dictation
• Text to Speech
Environment • Segmentation
• 3D Scanning
• Augmented Reality
• Background Removal
• 3D Object / Face / Room Scanning (Coming post-beta)
• 2D/3D Object Tracking
• Scene Perception (coming post-beta)

16. Understands Hardware Limits
16

17. Competitive
technologies focus
on a living-room
experience or a
sub-set of Intel
RealSense
technology
features
Designed for close-range interactions
17
120 cm
Intel®RealSense™
3D camera
56°(v) x 72° (v)
20 cm

18. Leap, RealSense, Kinect
18
2,5 cm 60 cm 2 m 4 m

19. Coordinate Systems
19
World coordinates Image coordinates

20. Vertical Rages
20
60cm
58cm
120cm
56°
20cm
17cm
70cm 35cm
73cm
Effective
Range
Gestures
Range
Effective
Range
3D Facial Traking
Range
2D Facial Traking
Range
115cm

21. Vertical Misalignment
21
56°

22. Horizontal Rages
22
60cm
87cm
120cm
72°
20cm
24cm
Effective
Range
Gestures
Range
Effective
Range
3D Facial Traking
Range
2D Facial Traking
Range
170cm
70cm 35cm
108cm
50cm

23. Capture Volumes
23
The user is performing a hand gesture outside of the capture volume.
The camera will not see this gesture

24. Evaluate different settings and environment
24

25. RealSense Camera use IR light and Sunlight can blind the
camera!!!
• Check exposition during all day
• Verify that there isn’t direct light on the camera
Indoor/Outdoor
25

26. RealSense isn’t a Rugged device:
• Check temperatures (+3/33°)
• Check humidity
Indoor/Outdoor (2)
26

27. Comfortable positions
Your users are not GORILLAS!!!
27

28. User posture may affect design of a gesture
28

29. Input variability
29

30. Gesture Tracking
30

31. Feedback, feedback, feedback,…
31
View of user:
• User Viewport
• User Overlay

32. Feedback, feedback, feedback,…
32

33. … where actions performed for some other purpose or
unconscious signs are interpreted in order to
influence/improve/facilitate the actors' future interaction or day-to-
day life (from Alan Dix)
• The interaction is not purposeful from the person side, but it is designed “to
happen”
• It “happens” in relation to signs which are not done for that (body temperature,
unconscious reactions such as blink rate, or unconscious aspects of activities
such as typing rate, vocabulary shifts (e.g. modal verbs), actions done for other
purposes, …
• It is designed for people acting
Manage Incidental Interaction
33

34. NetSense
Fast and simplified way to create RealSense Applications

35. demo
https://github.com/mvaloriani/NetSense/
PM> Install-Package NetSense
35

36. Intel RealSense
36

37. 37