Intel real sense handson

Tips and Trikes from
Real Case Studies
Matteo Valoriani
mvaloriani@gmail.com
@MatteoValoriani
#RealSense

PhD at Politecnico of Milano
CEO of Fifth Element
Speaker and Consultant
Intel Software Innovator: Perceptual Computing
Microsoft MVP on Kinect
Who I am…
2
mvaloriani@gmail.com
@MatteoValoriani
Intel® RealSense™ Hands-On Lab - Milan

3Intel® RealSense™ Hands-On Lab - Milan
Follow me on
Twitter or the
Kitten gets it:
@MatteoValoriani

You have to be a
magician…
or at least a
good illusionist

UI evolution
Intel® RealSense™ Hands-On Lab - Milan 5

Understands 4 basic types of input
Categories of
Input
Capabilities Features
Hands • Hand and Finger Tracking
• Gesture Recognition
• 22-point Hand and Finger Tracking
• 9 static and dynamic mid-air gestures
Face • Face Detection and
Tracking
• Multiple Face Detection and tracking
• 78-point Landmark Detection (facial features)
• Emotion Recognition (7 emotions, coming post-Beta)
• Pulse Estimation
• Face Recognition (Coming post-beta)
Speech • Speech Recognition • Command and Control
• Dictation
• Text to Speech
Environment • Segmentation
• 3D Scanning
• Augmented Reality
• Background Removal
• 3D Object / Face / Room Scanning (Coming post-beta)
• 2D/3D Object Tracking
• Scene Perception (coming post-beta)

Understands Hardware Limits

Competitive
technologies focus
on a living-room
experience or a
sub-set of Intel
RealSense
technology
features
Designed for close-range interactions
120 cm
Intel®RealSense™
3D camera
56°(v) x 72° (v)
20 cm

Leap, RealSense, Kinect
2,5 cm 60 cm 2 m 4 m

Vertical Rages
60cm
58cm
120cm
56°
20cm
17cm
70cm 35cm
73cm
Effective Range
Gestures Range
Effective Range
3D Facial Traking Range
115cm

Vertical Misalignment
56°

Horizontal Rages
60cm
87cm
120cm
72°
20cm
24cm
Effective Range
Gestures Range
Effective Range
170cm
70cm 35cm
108cm
50cm

Capture Volumes
The user is performing a hand gesture outside of the capture volume.
The camera will not see this gesture

Evaluate different settings and environment

RealSense Camera use IR light and Sunlight can blind the
camera!!!
• Check exposition during all day
• Verify that there isn’t direct light on the camera
Indoor/Outdoor

RealSense isn’t a Rugged device:
• Check temperatures (+3/33°)
• Check humidity
Indoor/Outdoor (2)

Comfortable positions
Your users are not GORILLAS!!!

User posture may affect design of a gesture

Input variability

Feedback, feedback, feedback,…
View of user:
• User Viewport
• User Overlay

Feedback, feedback, feedback,…

… where actions performed for some other purpose or unconscious
signs are interpreted in order to influence/improve/facilitate the
actors' future interaction or day-to-day life (from Alan Dix)
• The interaction is not purposeful from the person side, but it is designed “to
happen”
• It “happens” in relation to signs which are not done for that (body temperature,
unconscious reactions such as blink rate, or unconscious aspects of activities such
as typing rate, vocabulary shifts (e.g. modal verbs), actions done for other
purposes, …
• It is designed for people acting
Manage Incidental Interaction

Gestures should have a clear cognitive association with the semantic
functions they perform and the effects they achieve. Intuitiveness can be
enforced by appropriate interface and feedbacks.
The semantics of gestural patterns that belong to everyday life or
common task should be as consistent as possible to their “conventional”
meaning, but also take into account that intuitiveness is strongly
associated with users’ cultural background, general knowledge, and
experience.
Semantic intuitiveness

Gestural communication involves more muscles than keyboard interaction
or speech. Gestural commands must therefore be concise and quick, and
minimize user’s effort and physical stress.
Two types of muscular stress are known: static, the effort required
maintaining a posture for a fixed amount of time; dynamic, related to the
effort required to move a portion of the body through a trajectory.
Minimalize Fatigue

It must be easy for the user to learn how to perform and remember
gestures, minimizing the mental load of recalling movement trajectories
and associated actions.
The learning rate depends on tasks, user experience, skills, as well as the
size of the gesture language (more gestures decrease the learnability
rate).
Favor ease of learning (Learnability) 1/2

The gestures that are most natural, easy to learn and are immediately assimilated
by the user are those that belong to everyday life, or involve the least physical
effort. These gestures should be associated to the most frequent interactions.
Complex gestures can be more expressive and give more control, but have a
higher learnability burden.
Hence there is clearly a tension between design requirements, among which a
compromise must be made: naturalness of gestures, minimum size of the gesture
language, expressiveness and completeness of the gesture language.
Favor ease of learning (Learnability) 2/2

Users can perform unintended gestures, i.e., movements that are not
meant to communicate with the system they are interacting with.
The “immersion syndrome” occurs if every movement is interpreted by
the system, whether or not it was intended, and may determine
interaction effects against the user’s will.
Intentionality (Immersion Syndrome) 1/2

The designer must identify well-defined means to detect the intention of
the gestures, as distinguishing useful movements from unintentional ones
is not easy.
Body tension and non-relaxed posture of users can be used to make
explicit the user intention to start interaction, issue a command, or
confirm a choice.
The tense period should be short to not generate fatigue.
Intentionality (Immersion Syndrome) 2/2

Appropriate feedback indicating the effects and correctness of
the gesture performed is necessary for successful interaction,
and to improve the user's confidence in the system.
Not-self-revealing

Intel RealSense
Inspiration Projects

https://software.intel.com/sites/campaigns/perceptualshowcase/
winners.htm
http://www.intel.com/content/www/us/en/architecture-and-
technology/realsense-jim-parsons-flight-attendant-to-mars.html
Intel® Perceptual Computing Showcase

34
• The robot is built using Legos* Mindstorm
• A standard Ultrabook is place on top
• It acts as the brains
• As well as the interface
• Intel 3D Cameras are used to capture the
environment & steer the robot around
• It also allows for some facial & gesture recognition
Rover the
Self-Driving Car
Intel® RealSense™ Hands-On Lab - Milan

Space Between
• Used both Gesture & Voice commands in this
Puzzle Oriented Game about teaching a creature to
survive in a virtual world
• Used Unity* 3D as its game engine

3D Head Scanner
• Rapidly (about a minute) does a full scan of
the user’s head for avatar creation or other
uses such as fashion & beauty applications
• In the original app, users could change hair
styles, skin complexion, etc.
• Built as a Unity* project, it could be used by
other applications & games for a wild array of
different usage models

Stargate SG1
Gunship V2
• An evolution of the award
winning Perceptual Computing
game
• Controller-free, immersive UX
• Two-handed, multi-axis camera
and fire controls
• Menu and in-game voice
commands

Redwall: The
Warrior Reborn
• Facial landmark mapping to
character avatars
• Fully controller free UI
• Expression sensitive NPC
interactions.

Twitter Question!

WHAT IS THE EFFECTIVE
RANGE FOR 3D FACIAL
TRACKING?
#RealSense
@IntelRealSense

Intel real sense handson

More Related Content

What's hot

Similar to Intel real sense handson

More from Matteo Valoriani

Recently uploaded

Intel real sense handson