SlideShare a Scribd company logo
1 of 40
Download to read offline
Tips and Trikes from
Real Case Studies
Matteo Valoriani
mvaloriani@gmail.com
@MatteoValoriani
#RealSense
PhD at Politecnico of Milano
CEO of Fifth Element
Speaker and Consultant
Intel Software Innovator: Perceptual Computing
Microsoft MVP on Kinect
Who I am…
2
mvaloriani@gmail.com
@MatteoValoriani
Intel® RealSense™ Hands-On Lab - Milan
3Intel® RealSense™ Hands-On Lab - Milan
Follow me on
Twitter or the
Kitten gets it:
@MatteoValoriani
4Intel® RealSense™ Hands-On Lab - Milan
You have to be a
magician…
or at least a
good illusionist
UI evolution
Intel® RealSense™ Hands-On Lab - Milan 5
Understands 4 basic types of input
6Intel® RealSense™ Hands-On Lab - Milan
Categories of
Input
Capabilities Features
Hands • Hand and Finger Tracking
• Gesture Recognition
• 22-point Hand and Finger Tracking
• 9 static and dynamic mid-air gestures
Face • Face Detection and
Tracking
• Multiple Face Detection and tracking
• 78-point Landmark Detection (facial features)
• Emotion Recognition (7 emotions, coming post-Beta)
• Pulse Estimation
• Face Recognition (Coming post-beta)
Speech • Speech Recognition • Command and Control
• Dictation
• Text to Speech
Environment • Segmentation
• 3D Scanning
• Augmented Reality
• Background Removal
• 3D Object / Face / Room Scanning (Coming post-beta)
• 2D/3D Object Tracking
• Scene Perception (coming post-beta)
Understands Hardware Limits
7Intel® RealSense™ Hands-On Lab - Milan
Competitive
technologies focus
on a living-room
experience or a
sub-set of Intel
RealSense
technology
features
Designed for close-range interactions
8Intel® RealSense™ Hands-On Lab - Milan
120 cm
Intel®RealSense™
3D camera
56°(v) x 72° (v)
20 cm
Leap, RealSense, Kinect
9Intel® RealSense™ Hands-On Lab - Milan
2,5 cm 60 cm 2 m 4 m
Vertical Rages
10Intel® RealSense™ Hands-On Lab - Milan
60cm
58cm
120cm
56°
20cm
17cm
70cm 35cm
73cm
Effective Range
Gestures Range
Effective Range
3D Facial Traking Range
2D Facial Traking Range
115cm
Vertical Misalignment
11Intel® RealSense™ Hands-On Lab - Milan
56°
Horizontal Rages
12Intel® RealSense™ Hands-On Lab - Milan
60cm
87cm
120cm
72°
20cm
24cm
Effective Range
Gestures Range
Effective Range
3D Facial Traking Range
2D Facial Traking Range
170cm
70cm 35cm
108cm
50cm
Capture Volumes
13Intel® RealSense™ Hands-On Lab - Milan
The user is performing a hand gesture outside of the capture volume.
The camera will not see this gesture
Evaluate different settings and environment
14Intel® RealSense™ Hands-On Lab - Milan
RealSense Camera use IR light and Sunlight can blind the
camera!!!
• Check exposition during all day
• Verify that there isn’t direct light on the camera
Indoor/Outdoor
15Intel® RealSense™ Hands-On Lab - Milan
RealSense isn’t a Rugged device:
• Check temperatures (+3/33°)
• Check humidity
Indoor/Outdoor (2)
16Intel® RealSense™ Hands-On Lab - Milan
Comfortable positions
Your users are not GORILLAS!!!
Intel® RealSense™ Hands-On Lab - Milan 17
User posture may affect design of a gesture
Intel® RealSense™ Hands-On Lab - Milan 18
Input variability
Intel® RealSense™ Hands-On Lab - Milan 19
Feedback, feedback, feedback,…
20Intel® RealSense™ Hands-On Lab - Milan
View of user:
• User Viewport
• User Overlay
Feedback, feedback, feedback,…
21Intel® RealSense™ Hands-On Lab - Milan
… where actions performed for some other purpose or unconscious
signs are interpreted in order to influence/improve/facilitate the
actors' future interaction or day-to-day life (from Alan Dix)
• The interaction is not purposeful from the person side, but it is designed “to
happen”
• It “happens” in relation to signs which are not done for that (body temperature,
unconscious reactions such as blink rate, or unconscious aspects of activities such
as typing rate, vocabulary shifts (e.g. modal verbs), actions done for other
purposes, …
• It is designed for people acting
Manage Incidental Interaction
22Intel® RealSense™ Hands-On Lab - Milan
23Intel® RealSense™ Hands-On Lab - Milan
Takeaways
Gestures should have a clear cognitive association with the semantic
functions they perform and the effects they achieve. Intuitiveness can be
enforced by appropriate interface and feedbacks.
The semantics of gestural patterns that belong to everyday life or
common task should be as consistent as possible to their “conventional”
meaning, but also take into account that intuitiveness is strongly
associated with users’ cultural background, general knowledge, and
experience.
Semantic intuitiveness
Gestural communication involves more muscles than keyboard interaction
or speech. Gestural commands must therefore be concise and quick, and
minimize user’s effort and physical stress.
Two types of muscular stress are known: static, the effort required
maintaining a posture for a fixed amount of time; dynamic, related to the
effort required to move a portion of the body through a trajectory.
Minimalize Fatigue
It must be easy for the user to learn how to perform and remember
gestures, minimizing the mental load of recalling movement trajectories
and associated actions.
The learning rate depends on tasks, user experience, skills, as well as the
size of the gesture language (more gestures decrease the learnability
rate).
Favor ease of learning (Learnability) 1/2
The gestures that are most natural, easy to learn and are immediately assimilated
by the user are those that belong to everyday life, or involve the least physical
effort. These gestures should be associated to the most frequent interactions.
Complex gestures can be more expressive and give more control, but have a
higher learnability burden.
Hence there is clearly a tension between design requirements, among which a
compromise must be made: naturalness of gestures, minimum size of the gesture
language, expressiveness and completeness of the gesture language.
Favor ease of learning (Learnability) 2/2
Users can perform unintended gestures, i.e., movements that are not
meant to communicate with the system they are interacting with.
The “immersion syndrome” occurs if every movement is interpreted by
the system, whether or not it was intended, and may determine
interaction effects against the user’s will.
Intentionality (Immersion Syndrome) 1/2
The designer must identify well-defined means to detect the intention of
the gestures, as distinguishing useful movements from unintentional ones
is not easy.
Body tension and non-relaxed posture of users can be used to make
explicit the user intention to start interaction, issue a command, or
confirm a choice.
The tense period should be short to not generate fatigue.
Intentionality (Immersion Syndrome) 2/2
Appropriate feedback indicating the effects and correctness of
the gesture performed is necessary for successful interaction,
and to improve the user's confidence in the system.
Not-self-revealing
Intel RealSense
Inspiration Projects
32Intel® RealSense™ Hands-On Lab - Milan
https://software.intel.com/sites/campaigns/perceptualshowcase/
winners.htm
http://www.intel.com/content/www/us/en/architecture-and-
technology/realsense-jim-parsons-flight-attendant-to-mars.html
Intel® Perceptual Computing Showcase
33Intel® RealSense™ Hands-On Lab - Milan
34
• The robot is built using Legos* Mindstorm
• A standard Ultrabook is place on top
• It acts as the brains
• As well as the interface
• Intel 3D Cameras are used to capture the
environment & steer the robot around
• It also allows for some facial & gesture recognition
Rover the
Self-Driving Car
Intel® RealSense™ Hands-On Lab - Milan
Space Between
• Used both Gesture & Voice commands in this
Puzzle Oriented Game about teaching a creature to
survive in a virtual world
• Used Unity* 3D as its game engine
35Intel® RealSense™ Hands-On Lab - Milan
3D Head Scanner
• Rapidly (about a minute) does a full scan of
the user’s head for avatar creation or other
uses such as fashion & beauty applications
• In the original app, users could change hair
styles, skin complexion, etc.
• Built as a Unity* project, it could be used by
other applications & games for a wild array of
different usage models
36Intel® RealSense™ Hands-On Lab - Milan
Stargate SG1
Gunship V2
• An evolution of the award
winning Perceptual Computing
game
• Controller-free, immersive UX
• Two-handed, multi-axis camera
and fire controls
• Menu and in-game voice
commands
37Intel® RealSense™ Hands-On Lab - Milan
Redwall: The
Warrior Reborn
• Facial landmark mapping to
character avatars
• Fully controller free UI
• Expression sensitive NPC
interactions.
38Intel® RealSense™ Hands-On Lab - Milan
Twitter Question!
39Intel® RealSense™ Hands-On Lab - Milan
WHAT IS THE EFFECTIVE
RANGE FOR 3D FACIAL
TRACKING?
40Intel® RealSense™ Hands-On Lab - Milan
#RealSense
@IntelRealSense

More Related Content

What's hot

Intel® RealSense™ Technology Adding Human-Like Sensing to Computing Devices
Intel® RealSense™ Technology Adding Human-Like Sensing to Computing DevicesIntel® RealSense™ Technology Adding Human-Like Sensing to Computing Devices
Intel® RealSense™ Technology Adding Human-Like Sensing to Computing DevicesIntel® Software
 
Intel® RealSense™ Technology: Code Walk-through Presented by Intel Software I...
Intel® RealSense™ Technology: Code Walk-through Presented by Intel Software I...Intel® RealSense™ Technology: Code Walk-through Presented by Intel Software I...
Intel® RealSense™ Technology: Code Walk-through Presented by Intel Software I...Intel® Software
 
Dev09 – la battaglia del touchless
Dev09 – la battaglia del touchlessDev09 – la battaglia del touchless
Dev09 – la battaglia del touchlessClemente Giorio
 
Etna dev 2016 - Deep Dive Holographic Applications
Etna dev 2016 - Deep Dive Holographic ApplicationsEtna dev 2016 - Deep Dive Holographic Applications
Etna dev 2016 - Deep Dive Holographic ApplicationsMatteo Valoriani
 
Microsoft Kinect and Kinect SDK
Microsoft Kinect and Kinect SDKMicrosoft Kinect and Kinect SDK
Microsoft Kinect and Kinect SDKSiraj Memon
 
Writing applications using the Microsoft Kinect Sensor
Writing applications using the Microsoft Kinect SensorWriting applications using the Microsoft Kinect Sensor
Writing applications using the Microsoft Kinect Sensorphildenoncourt
 
Web cam sensing using sdk tool
Web cam sensing using sdk tool Web cam sensing using sdk tool
Web cam sensing using sdk tool eSAT Journals
 
Touchless technology
Touchless technologyTouchless technology
Touchless technologyInternet User
 
Touchless touchscreen
Touchless touchscreenTouchless touchscreen
Touchless touchscreenNaga Dinesh
 
Intel RealSense & Perceptual Computing
Intel RealSense & Perceptual ComputingIntel RealSense & Perceptual Computing
Intel RealSense & Perceptual ComputingFrancesca Tosi
 
Natural User Interfaces
Natural User InterfacesNatural User Interfaces
Natural User InterfacesLuigi Oliveto
 
Introduction to Google Project Tango and Intel® RealSense™
Introduction to Google Project Tango and Intel® RealSense™Introduction to Google Project Tango and Intel® RealSense™
Introduction to Google Project Tango and Intel® RealSense™Francesca Tosi
 
Softkinetic user interface evolution by Ilse Ravyse and Tanya Varbanove
Softkinetic user interface evolution by Ilse Ravyse and Tanya VarbanoveSoftkinetic user interface evolution by Ilse Ravyse and Tanya Varbanove
Softkinetic user interface evolution by Ilse Ravyse and Tanya VarbanoveIndustrial Design Center
 
Touchless technology Seminar Presentation
Touchless technology Seminar PresentationTouchless technology Seminar Presentation
Touchless technology Seminar PresentationAparna Nk
 
Touchless Touchscreen
Touchless TouchscreenTouchless Touchscreen
Touchless TouchscreenTasnin Khan
 
Microsoft Kinect in Healthcare
Microsoft Kinect in HealthcareMicrosoft Kinect in Healthcare
Microsoft Kinect in HealthcareGSW
 
Natural User Interface Microsoft Kinect and Surface Computing
Natural User Interface Microsoft Kinect and Surface ComputingNatural User Interface Microsoft Kinect and Surface Computing
Natural User Interface Microsoft Kinect and Surface ComputingYuvaraj Ilangovan
 

What's hot (20)

Intel® RealSense™ Technology Adding Human-Like Sensing to Computing Devices
Intel® RealSense™ Technology Adding Human-Like Sensing to Computing DevicesIntel® RealSense™ Technology Adding Human-Like Sensing to Computing Devices
Intel® RealSense™ Technology Adding Human-Like Sensing to Computing Devices
 
Intel® RealSense™ Technology: Code Walk-through Presented by Intel Software I...
Intel® RealSense™ Technology: Code Walk-through Presented by Intel Software I...Intel® RealSense™ Technology: Code Walk-through Presented by Intel Software I...
Intel® RealSense™ Technology: Code Walk-through Presented by Intel Software I...
 
Dev09 – la battaglia del touchless
Dev09 – la battaglia del touchlessDev09 – la battaglia del touchless
Dev09 – la battaglia del touchless
 
Communitydays2014
Communitydays2014Communitydays2014
Communitydays2014
 
Etna dev 2016 - Deep Dive Holographic Applications
Etna dev 2016 - Deep Dive Holographic ApplicationsEtna dev 2016 - Deep Dive Holographic Applications
Etna dev 2016 - Deep Dive Holographic Applications
 
Touch Less touch screen
Touch Less touch screenTouch Less touch screen
Touch Less touch screen
 
Microsoft Kinect and Kinect SDK
Microsoft Kinect and Kinect SDKMicrosoft Kinect and Kinect SDK
Microsoft Kinect and Kinect SDK
 
Writing applications using the Microsoft Kinect Sensor
Writing applications using the Microsoft Kinect SensorWriting applications using the Microsoft Kinect Sensor
Writing applications using the Microsoft Kinect Sensor
 
Web cam sensing using sdk tool
Web cam sensing using sdk tool Web cam sensing using sdk tool
Web cam sensing using sdk tool
 
Touchless technology
Touchless technologyTouchless technology
Touchless technology
 
Touchless touchscreen
Touchless touchscreenTouchless touchscreen
Touchless touchscreen
 
Intel RealSense & Perceptual Computing
Intel RealSense & Perceptual ComputingIntel RealSense & Perceptual Computing
Intel RealSense & Perceptual Computing
 
Natural User Interfaces
Natural User InterfacesNatural User Interfaces
Natural User Interfaces
 
Introduction to Google Project Tango and Intel® RealSense™
Introduction to Google Project Tango and Intel® RealSense™Introduction to Google Project Tango and Intel® RealSense™
Introduction to Google Project Tango and Intel® RealSense™
 
Softkinetic user interface evolution by Ilse Ravyse and Tanya Varbanove
Softkinetic user interface evolution by Ilse Ravyse and Tanya VarbanoveSoftkinetic user interface evolution by Ilse Ravyse and Tanya Varbanove
Softkinetic user interface evolution by Ilse Ravyse and Tanya Varbanove
 
Touchless technology Seminar Presentation
Touchless technology Seminar PresentationTouchless technology Seminar Presentation
Touchless technology Seminar Presentation
 
Touchless Touchscreen
Touchless TouchscreenTouchless Touchscreen
Touchless Touchscreen
 
Touchless Touch screen technology
Touchless Touch screen technologyTouchless Touch screen technology
Touchless Touch screen technology
 
Microsoft Kinect in Healthcare
Microsoft Kinect in HealthcareMicrosoft Kinect in Healthcare
Microsoft Kinect in Healthcare
 
Natural User Interface Microsoft Kinect and Surface Computing
Natural User Interface Microsoft Kinect and Surface ComputingNatural User Interface Microsoft Kinect and Surface Computing
Natural User Interface Microsoft Kinect and Surface Computing
 

Similar to Intel real sense handson

Gesture Recogntion Technology
Gesture Recogntion TechnologyGesture Recogntion Technology
Gesture Recogntion TechnologyMohit Sipani
 
ppt of gesture recognition
ppt of gesture recognitionppt of gesture recognition
ppt of gesture recognitionAayush Agrawal
 
Microinteractions - Annotated Portfolio
Microinteractions - Annotated PortfolioMicrointeractions - Annotated Portfolio
Microinteractions - Annotated PortfolioSaroj Tailor
 
IRJET- Hand Gesture Recognition for Deaf and Dumb
IRJET- Hand Gesture Recognition for Deaf and DumbIRJET- Hand Gesture Recognition for Deaf and Dumb
IRJET- Hand Gesture Recognition for Deaf and DumbIRJET Journal
 
Touchless-Touchscreen-Technology for electronics nd communictions.pptx
Touchless-Touchscreen-Technology for electronics nd communictions.pptxTouchless-Touchscreen-Technology for electronics nd communictions.pptx
Touchless-Touchscreen-Technology for electronics nd communictions.pptxKavya158188
 
Gesture Recognition Technology
Gesture Recognition TechnologyGesture Recognition Technology
Gesture Recognition TechnologyNikith Kumar Reddy
 
Touchscreen UX Design Workshop
Touchscreen UX Design WorkshopTouchscreen UX Design Workshop
Touchscreen UX Design WorkshopKirsten Miller
 
Touchless touchscreen technology
Touchless touchscreen technologyTouchless touchscreen technology
Touchless touchscreen technologyMATHEW JOSEPH
 
Touchless touchscreen technology
Touchless touchscreen technologyTouchless touchscreen technology
Touchless touchscreen technologyMATHEW JOSEPH
 
A computer vision based virtual mouse
A computer vision based virtual mouseA computer vision based virtual mouse
A computer vision based virtual mouseStudentRocks
 
A Survey on Detecting Hand Gesture
A Survey on Detecting Hand GestureA Survey on Detecting Hand Gesture
A Survey on Detecting Hand GestureIRJET Journal
 
Blue Eyes Technology
Blue Eyes TechnologyBlue Eyes Technology
Blue Eyes TechnologyColloquium
 
HCI Unit 3.pptx
HCI Unit 3.pptxHCI Unit 3.pptx
HCI Unit 3.pptxRaja980775
 

Similar to Intel real sense handson (20)

Gesture Recogntion Technology
Gesture Recogntion TechnologyGesture Recogntion Technology
Gesture Recogntion Technology
 
UX 101
UX 101UX 101
UX 101
 
Touchless Touchscreen
Touchless TouchscreenTouchless Touchscreen
Touchless Touchscreen
 
Sixth Sense Technology
Sixth Sense TechnologySixth Sense Technology
Sixth Sense Technology
 
Gesture controlled device
Gesture controlled deviceGesture controlled device
Gesture controlled device
 
ppt of gesture recognition
ppt of gesture recognitionppt of gesture recognition
ppt of gesture recognition
 
Niknewppt
NiknewpptNiknewppt
Niknewppt
 
Microinteractions - Annotated Portfolio
Microinteractions - Annotated PortfolioMicrointeractions - Annotated Portfolio
Microinteractions - Annotated Portfolio
 
IRJET- Hand Gesture Recognition for Deaf and Dumb
IRJET- Hand Gesture Recognition for Deaf and DumbIRJET- Hand Gesture Recognition for Deaf and Dumb
IRJET- Hand Gesture Recognition for Deaf and Dumb
 
Touchless-Touchscreen-Technology for electronics nd communictions.pptx
Touchless-Touchscreen-Technology for electronics nd communictions.pptxTouchless-Touchscreen-Technology for electronics nd communictions.pptx
Touchless-Touchscreen-Technology for electronics nd communictions.pptx
 
Nikppt
NikpptNikppt
Nikppt
 
Gesture Recognition Technology
Gesture Recognition TechnologyGesture Recognition Technology
Gesture Recognition Technology
 
Touchscreen UX Design Workshop
Touchscreen UX Design WorkshopTouchscreen UX Design Workshop
Touchscreen UX Design Workshop
 
Touchless touchscreen technology
Touchless touchscreen technologyTouchless touchscreen technology
Touchless touchscreen technology
 
Touchless touchscreen technology
Touchless touchscreen technologyTouchless touchscreen technology
Touchless touchscreen technology
 
A computer vision based virtual mouse
A computer vision based virtual mouseA computer vision based virtual mouse
A computer vision based virtual mouse
 
Gesture Recognition
Gesture RecognitionGesture Recognition
Gesture Recognition
 
A Survey on Detecting Hand Gesture
A Survey on Detecting Hand GestureA Survey on Detecting Hand Gesture
A Survey on Detecting Hand Gesture
 
Blue Eyes Technology
Blue Eyes TechnologyBlue Eyes Technology
Blue Eyes Technology
 
HCI Unit 3.pptx
HCI Unit 3.pptxHCI Unit 3.pptx
HCI Unit 3.pptx
 

More from Matteo Valoriani

Immerge yourself in a new Reality
Immerge yourself in a new RealityImmerge yourself in a new Reality
Immerge yourself in a new RealityMatteo Valoriani
 
How Augment your Reality: Different perspective on the Reality / Virtuality C...
How Augment your Reality: Different perspective on the Reality / Virtuality C...How Augment your Reality: Different perspective on the Reality / Virtuality C...
How Augment your Reality: Different perspective on the Reality / Virtuality C...Matteo Valoriani
 
Debug, Analyze and Optimize Games with Intel Tools
Debug, Analyze and Optimize Games with Intel Tools Debug, Analyze and Optimize Games with Intel Tools
Debug, Analyze and Optimize Games with Intel Tools Matteo Valoriani
 
More Personal Computing: Windows 10, Kinect and Wearables
More Personal Computing: Windows 10, Kinect and WearablesMore Personal Computing: Windows 10, Kinect and Wearables
More Personal Computing: Windows 10, Kinect and WearablesMatteo Valoriani
 
Introduction to development
Introduction to developmentIntroduction to development
Introduction to developmentMatteo Valoriani
 
Etna dev 2016 - Introduction to Holographic Development
Etna dev 2016 - Introduction to Holographic DevelopmentEtna dev 2016 - Introduction to Holographic Development
Etna dev 2016 - Introduction to Holographic DevelopmentMatteo Valoriani
 
Etna dev 2016 - Introduction to Mixed Reality with HoloLens
Etna dev 2016 - Introduction to Mixed Reality with HoloLensEtna dev 2016 - Introduction to Mixed Reality with HoloLens
Etna dev 2016 - Introduction to Mixed Reality with HoloLensMatteo Valoriani
 
Mixed Reality from demo to product
Mixed Reality from demo to productMixed Reality from demo to product
Mixed Reality from demo to productMatteo Valoriani
 
Develop store apps with kinect for windows v2
Develop store apps with kinect for windows v2Develop store apps with kinect for windows v2
Develop store apps with kinect for windows v2Matteo Valoriani
 
Programming with kinect v2
Programming with kinect v2Programming with kinect v2
Programming with kinect v2Matteo Valoriani
 
Tecnologie e Startup: ICT è solo una commodity?
Tecnologie e Startup: ICT è solo una commodity? Tecnologie e Startup: ICT è solo una commodity?
Tecnologie e Startup: ICT è solo una commodity? Matteo Valoriani
 
Corso pratico di C# - 2013
Corso pratico di C# - 2013Corso pratico di C# - 2013
Corso pratico di C# - 2013Matteo Valoriani
 
Introduction to Kinect - Update v 1.8
Introduction to Kinect - Update v 1.8Introduction to Kinect - Update v 1.8
Introduction to Kinect - Update v 1.8Matteo Valoriani
 
Smart and beyond - Perchè
Smart and beyond - PerchèSmart and beyond - Perchè
Smart and beyond - PerchèMatteo Valoriani
 
Uxconf2012 - Interactive technologies for children: new frontiers
Uxconf2012 - Interactive technologies for children: new frontiersUxconf2012 - Interactive technologies for children: new frontiers
Uxconf2012 - Interactive technologies for children: new frontiersMatteo Valoriani
 
6 track kinect@Bicocca - iniziative
6   track kinect@Bicocca - iniziative6   track kinect@Bicocca - iniziative
6 track kinect@Bicocca - iniziativeMatteo Valoriani
 
5 track kinect@Bicocca - gesture
5   track kinect@Bicocca - gesture5   track kinect@Bicocca - gesture
5 track kinect@Bicocca - gestureMatteo Valoriani
 

More from Matteo Valoriani (20)

Immerge yourself in a new Reality
Immerge yourself in a new RealityImmerge yourself in a new Reality
Immerge yourself in a new Reality
 
How Augment your Reality: Different perspective on the Reality / Virtuality C...
How Augment your Reality: Different perspective on the Reality / Virtuality C...How Augment your Reality: Different perspective on the Reality / Virtuality C...
How Augment your Reality: Different perspective on the Reality / Virtuality C...
 
Hour ofcode
Hour ofcodeHour ofcode
Hour ofcode
 
Debug, Analyze and Optimize Games with Intel Tools
Debug, Analyze and Optimize Games with Intel Tools Debug, Analyze and Optimize Games with Intel Tools
Debug, Analyze and Optimize Games with Intel Tools
 
More Personal Computing: Windows 10, Kinect and Wearables
More Personal Computing: Windows 10, Kinect and WearablesMore Personal Computing: Windows 10, Kinect and Wearables
More Personal Computing: Windows 10, Kinect and Wearables
 
Introduction to development
Introduction to developmentIntroduction to development
Introduction to development
 
Etna dev 2016 - Introduction to Holographic Development
Etna dev 2016 - Introduction to Holographic DevelopmentEtna dev 2016 - Introduction to Holographic Development
Etna dev 2016 - Introduction to Holographic Development
 
Etna dev 2016 - Introduction to Mixed Reality with HoloLens
Etna dev 2016 - Introduction to Mixed Reality with HoloLensEtna dev 2016 - Introduction to Mixed Reality with HoloLens
Etna dev 2016 - Introduction to Mixed Reality with HoloLens
 
Mixed Reality from demo to product
Mixed Reality from demo to productMixed Reality from demo to product
Mixed Reality from demo to product
 
Develop store apps with kinect for windows v2
Develop store apps with kinect for windows v2Develop store apps with kinect for windows v2
Develop store apps with kinect for windows v2
 
Programming with kinect v2
Programming with kinect v2Programming with kinect v2
Programming with kinect v2
 
Face recognition
Face recognitionFace recognition
Face recognition
 
Tecnologie e Startup: ICT è solo una commodity?
Tecnologie e Startup: ICT è solo una commodity? Tecnologie e Startup: ICT è solo una commodity?
Tecnologie e Startup: ICT è solo una commodity?
 
Communityday2013
Communityday2013Communityday2013
Communityday2013
 
Corso pratico di C# - 2013
Corso pratico di C# - 2013Corso pratico di C# - 2013
Corso pratico di C# - 2013
 
Introduction to Kinect - Update v 1.8
Introduction to Kinect - Update v 1.8Introduction to Kinect - Update v 1.8
Introduction to Kinect - Update v 1.8
 
Smart and beyond - Perchè
Smart and beyond - PerchèSmart and beyond - Perchè
Smart and beyond - Perchè
 
Uxconf2012 - Interactive technologies for children: new frontiers
Uxconf2012 - Interactive technologies for children: new frontiersUxconf2012 - Interactive technologies for children: new frontiers
Uxconf2012 - Interactive technologies for children: new frontiers
 
6 track kinect@Bicocca - iniziative
6   track kinect@Bicocca - iniziative6   track kinect@Bicocca - iniziative
6 track kinect@Bicocca - iniziative
 
5 track kinect@Bicocca - gesture
5   track kinect@Bicocca - gesture5   track kinect@Bicocca - gesture
5 track kinect@Bicocca - gesture
 

Recently uploaded

Artificial Intelligence & SEO Trends for 2024
Artificial Intelligence & SEO Trends for 2024Artificial Intelligence & SEO Trends for 2024
Artificial Intelligence & SEO Trends for 2024D Cloud Solutions
 
Secure your environment with UiPath and CyberArk technologies - Session 1
Secure your environment with UiPath and CyberArk technologies - Session 1Secure your environment with UiPath and CyberArk technologies - Session 1
Secure your environment with UiPath and CyberArk technologies - Session 1DianaGray10
 
NIST Cybersecurity Framework (CSF) 2.0 Workshop
NIST Cybersecurity Framework (CSF) 2.0 WorkshopNIST Cybersecurity Framework (CSF) 2.0 Workshop
NIST Cybersecurity Framework (CSF) 2.0 WorkshopBachir Benyammi
 
How Accurate are Carbon Emissions Projections?
How Accurate are Carbon Emissions Projections?How Accurate are Carbon Emissions Projections?
How Accurate are Carbon Emissions Projections?IES VE
 
AI Fame Rush Review – Virtual Influencer Creation In Just Minutes
AI Fame Rush Review – Virtual Influencer Creation In Just MinutesAI Fame Rush Review – Virtual Influencer Creation In Just Minutes
AI Fame Rush Review – Virtual Influencer Creation In Just MinutesMd Hossain Ali
 
Cybersecurity Workshop #1.pptx
Cybersecurity Workshop #1.pptxCybersecurity Workshop #1.pptx
Cybersecurity Workshop #1.pptxGDSC PJATK
 
activity_diagram_combine_v4_20190827.pdfactivity_diagram_combine_v4_20190827.pdf
activity_diagram_combine_v4_20190827.pdfactivity_diagram_combine_v4_20190827.pdfactivity_diagram_combine_v4_20190827.pdfactivity_diagram_combine_v4_20190827.pdf
activity_diagram_combine_v4_20190827.pdfactivity_diagram_combine_v4_20190827.pdfJamie (Taka) Wang
 
Anypoint Code Builder , Google Pub sub connector and MuleSoft RPA
Anypoint Code Builder , Google Pub sub connector and MuleSoft RPAAnypoint Code Builder , Google Pub sub connector and MuleSoft RPA
Anypoint Code Builder , Google Pub sub connector and MuleSoft RPAshyamraj55
 
UiPath Studio Web workshop series - Day 8
UiPath Studio Web workshop series - Day 8UiPath Studio Web workshop series - Day 8
UiPath Studio Web workshop series - Day 8DianaGray10
 
20230202 - Introduction to tis-py
20230202 - Introduction to tis-py20230202 - Introduction to tis-py
20230202 - Introduction to tis-pyJamie (Taka) Wang
 
Building AI-Driven Apps Using Semantic Kernel.pptx
Building AI-Driven Apps Using Semantic Kernel.pptxBuilding AI-Driven Apps Using Semantic Kernel.pptx
Building AI-Driven Apps Using Semantic Kernel.pptxUdaiappa Ramachandran
 
VoIP Service and Marketing using Odoo and Asterisk PBX
VoIP Service and Marketing using Odoo and Asterisk PBXVoIP Service and Marketing using Odoo and Asterisk PBX
VoIP Service and Marketing using Odoo and Asterisk PBXTarek Kalaji
 
Comparing Sidecar-less Service Mesh from Cilium and Istio
Comparing Sidecar-less Service Mesh from Cilium and IstioComparing Sidecar-less Service Mesh from Cilium and Istio
Comparing Sidecar-less Service Mesh from Cilium and IstioChristian Posta
 
OpenShift Commons Paris - Choose Your Own Observability Adventure
OpenShift Commons Paris - Choose Your Own Observability AdventureOpenShift Commons Paris - Choose Your Own Observability Adventure
OpenShift Commons Paris - Choose Your Own Observability AdventureEric D. Schabell
 
Igniting Next Level Productivity with AI-Infused Data Integration Workflows
Igniting Next Level Productivity with AI-Infused Data Integration WorkflowsIgniting Next Level Productivity with AI-Infused Data Integration Workflows
Igniting Next Level Productivity with AI-Infused Data Integration WorkflowsSafe Software
 
UWB Technology for Enhanced Indoor and Outdoor Positioning in Physiological M...
UWB Technology for Enhanced Indoor and Outdoor Positioning in Physiological M...UWB Technology for Enhanced Indoor and Outdoor Positioning in Physiological M...
UWB Technology for Enhanced Indoor and Outdoor Positioning in Physiological M...UbiTrack UK
 
Meet the new FSP 3000 M-Flex800™
Meet the new FSP 3000 M-Flex800™Meet the new FSP 3000 M-Flex800™
Meet the new FSP 3000 M-Flex800™Adtran
 
UiPath Studio Web workshop series - Day 6
UiPath Studio Web workshop series - Day 6UiPath Studio Web workshop series - Day 6
UiPath Studio Web workshop series - Day 6DianaGray10
 
IESVE Software for Florida Code Compliance Using ASHRAE 90.1-2019
IESVE Software for Florida Code Compliance Using ASHRAE 90.1-2019IESVE Software for Florida Code Compliance Using ASHRAE 90.1-2019
IESVE Software for Florida Code Compliance Using ASHRAE 90.1-2019IES VE
 
Building Your Own AI Instance (TBLC AI )
Building Your Own AI Instance (TBLC AI )Building Your Own AI Instance (TBLC AI )
Building Your Own AI Instance (TBLC AI )Brian Pichman
 

Recently uploaded (20)

Artificial Intelligence & SEO Trends for 2024
Artificial Intelligence & SEO Trends for 2024Artificial Intelligence & SEO Trends for 2024
Artificial Intelligence & SEO Trends for 2024
 
Secure your environment with UiPath and CyberArk technologies - Session 1
Secure your environment with UiPath and CyberArk technologies - Session 1Secure your environment with UiPath and CyberArk technologies - Session 1
Secure your environment with UiPath and CyberArk technologies - Session 1
 
NIST Cybersecurity Framework (CSF) 2.0 Workshop
NIST Cybersecurity Framework (CSF) 2.0 WorkshopNIST Cybersecurity Framework (CSF) 2.0 Workshop
NIST Cybersecurity Framework (CSF) 2.0 Workshop
 
How Accurate are Carbon Emissions Projections?
How Accurate are Carbon Emissions Projections?How Accurate are Carbon Emissions Projections?
How Accurate are Carbon Emissions Projections?
 
AI Fame Rush Review – Virtual Influencer Creation In Just Minutes
AI Fame Rush Review – Virtual Influencer Creation In Just MinutesAI Fame Rush Review – Virtual Influencer Creation In Just Minutes
AI Fame Rush Review – Virtual Influencer Creation In Just Minutes
 
Cybersecurity Workshop #1.pptx
Cybersecurity Workshop #1.pptxCybersecurity Workshop #1.pptx
Cybersecurity Workshop #1.pptx
 
activity_diagram_combine_v4_20190827.pdfactivity_diagram_combine_v4_20190827.pdf
activity_diagram_combine_v4_20190827.pdfactivity_diagram_combine_v4_20190827.pdfactivity_diagram_combine_v4_20190827.pdfactivity_diagram_combine_v4_20190827.pdf
activity_diagram_combine_v4_20190827.pdfactivity_diagram_combine_v4_20190827.pdf
 
Anypoint Code Builder , Google Pub sub connector and MuleSoft RPA
Anypoint Code Builder , Google Pub sub connector and MuleSoft RPAAnypoint Code Builder , Google Pub sub connector and MuleSoft RPA
Anypoint Code Builder , Google Pub sub connector and MuleSoft RPA
 
UiPath Studio Web workshop series - Day 8
UiPath Studio Web workshop series - Day 8UiPath Studio Web workshop series - Day 8
UiPath Studio Web workshop series - Day 8
 
20230202 - Introduction to tis-py
20230202 - Introduction to tis-py20230202 - Introduction to tis-py
20230202 - Introduction to tis-py
 
Building AI-Driven Apps Using Semantic Kernel.pptx
Building AI-Driven Apps Using Semantic Kernel.pptxBuilding AI-Driven Apps Using Semantic Kernel.pptx
Building AI-Driven Apps Using Semantic Kernel.pptx
 
VoIP Service and Marketing using Odoo and Asterisk PBX
VoIP Service and Marketing using Odoo and Asterisk PBXVoIP Service and Marketing using Odoo and Asterisk PBX
VoIP Service and Marketing using Odoo and Asterisk PBX
 
Comparing Sidecar-less Service Mesh from Cilium and Istio
Comparing Sidecar-less Service Mesh from Cilium and IstioComparing Sidecar-less Service Mesh from Cilium and Istio
Comparing Sidecar-less Service Mesh from Cilium and Istio
 
OpenShift Commons Paris - Choose Your Own Observability Adventure
OpenShift Commons Paris - Choose Your Own Observability AdventureOpenShift Commons Paris - Choose Your Own Observability Adventure
OpenShift Commons Paris - Choose Your Own Observability Adventure
 
Igniting Next Level Productivity with AI-Infused Data Integration Workflows
Igniting Next Level Productivity with AI-Infused Data Integration WorkflowsIgniting Next Level Productivity with AI-Infused Data Integration Workflows
Igniting Next Level Productivity with AI-Infused Data Integration Workflows
 
UWB Technology for Enhanced Indoor and Outdoor Positioning in Physiological M...
UWB Technology for Enhanced Indoor and Outdoor Positioning in Physiological M...UWB Technology for Enhanced Indoor and Outdoor Positioning in Physiological M...
UWB Technology for Enhanced Indoor and Outdoor Positioning in Physiological M...
 
Meet the new FSP 3000 M-Flex800™
Meet the new FSP 3000 M-Flex800™Meet the new FSP 3000 M-Flex800™
Meet the new FSP 3000 M-Flex800™
 
UiPath Studio Web workshop series - Day 6
UiPath Studio Web workshop series - Day 6UiPath Studio Web workshop series - Day 6
UiPath Studio Web workshop series - Day 6
 
IESVE Software for Florida Code Compliance Using ASHRAE 90.1-2019
IESVE Software for Florida Code Compliance Using ASHRAE 90.1-2019IESVE Software for Florida Code Compliance Using ASHRAE 90.1-2019
IESVE Software for Florida Code Compliance Using ASHRAE 90.1-2019
 
Building Your Own AI Instance (TBLC AI )
Building Your Own AI Instance (TBLC AI )Building Your Own AI Instance (TBLC AI )
Building Your Own AI Instance (TBLC AI )
 

Intel real sense handson

  • 1. Tips and Trikes from Real Case Studies Matteo Valoriani mvaloriani@gmail.com @MatteoValoriani #RealSense
  • 2. PhD at Politecnico of Milano CEO of Fifth Element Speaker and Consultant Intel Software Innovator: Perceptual Computing Microsoft MVP on Kinect Who I am… 2 mvaloriani@gmail.com @MatteoValoriani Intel® RealSense™ Hands-On Lab - Milan
  • 3. 3Intel® RealSense™ Hands-On Lab - Milan Follow me on Twitter or the Kitten gets it: @MatteoValoriani
  • 4. 4Intel® RealSense™ Hands-On Lab - Milan You have to be a magician… or at least a good illusionist
  • 5. UI evolution Intel® RealSense™ Hands-On Lab - Milan 5
  • 6. Understands 4 basic types of input 6Intel® RealSense™ Hands-On Lab - Milan Categories of Input Capabilities Features Hands • Hand and Finger Tracking • Gesture Recognition • 22-point Hand and Finger Tracking • 9 static and dynamic mid-air gestures Face • Face Detection and Tracking • Multiple Face Detection and tracking • 78-point Landmark Detection (facial features) • Emotion Recognition (7 emotions, coming post-Beta) • Pulse Estimation • Face Recognition (Coming post-beta) Speech • Speech Recognition • Command and Control • Dictation • Text to Speech Environment • Segmentation • 3D Scanning • Augmented Reality • Background Removal • 3D Object / Face / Room Scanning (Coming post-beta) • 2D/3D Object Tracking • Scene Perception (coming post-beta)
  • 7. Understands Hardware Limits 7Intel® RealSense™ Hands-On Lab - Milan
  • 8. Competitive technologies focus on a living-room experience or a sub-set of Intel RealSense technology features Designed for close-range interactions 8Intel® RealSense™ Hands-On Lab - Milan 120 cm Intel®RealSense™ 3D camera 56°(v) x 72° (v) 20 cm
  • 9. Leap, RealSense, Kinect 9Intel® RealSense™ Hands-On Lab - Milan 2,5 cm 60 cm 2 m 4 m
  • 10. Vertical Rages 10Intel® RealSense™ Hands-On Lab - Milan 60cm 58cm 120cm 56° 20cm 17cm 70cm 35cm 73cm Effective Range Gestures Range Effective Range 3D Facial Traking Range 2D Facial Traking Range 115cm
  • 11. Vertical Misalignment 11Intel® RealSense™ Hands-On Lab - Milan 56°
  • 12. Horizontal Rages 12Intel® RealSense™ Hands-On Lab - Milan 60cm 87cm 120cm 72° 20cm 24cm Effective Range Gestures Range Effective Range 3D Facial Traking Range 2D Facial Traking Range 170cm 70cm 35cm 108cm 50cm
  • 13. Capture Volumes 13Intel® RealSense™ Hands-On Lab - Milan The user is performing a hand gesture outside of the capture volume. The camera will not see this gesture
  • 14. Evaluate different settings and environment 14Intel® RealSense™ Hands-On Lab - Milan
  • 15. RealSense Camera use IR light and Sunlight can blind the camera!!! • Check exposition during all day • Verify that there isn’t direct light on the camera Indoor/Outdoor 15Intel® RealSense™ Hands-On Lab - Milan
  • 16. RealSense isn’t a Rugged device: • Check temperatures (+3/33°) • Check humidity Indoor/Outdoor (2) 16Intel® RealSense™ Hands-On Lab - Milan
  • 17. Comfortable positions Your users are not GORILLAS!!! Intel® RealSense™ Hands-On Lab - Milan 17
  • 18. User posture may affect design of a gesture Intel® RealSense™ Hands-On Lab - Milan 18
  • 19. Input variability Intel® RealSense™ Hands-On Lab - Milan 19
  • 20. Feedback, feedback, feedback,… 20Intel® RealSense™ Hands-On Lab - Milan View of user: • User Viewport • User Overlay
  • 21. Feedback, feedback, feedback,… 21Intel® RealSense™ Hands-On Lab - Milan
  • 22. … where actions performed for some other purpose or unconscious signs are interpreted in order to influence/improve/facilitate the actors' future interaction or day-to-day life (from Alan Dix) • The interaction is not purposeful from the person side, but it is designed “to happen” • It “happens” in relation to signs which are not done for that (body temperature, unconscious reactions such as blink rate, or unconscious aspects of activities such as typing rate, vocabulary shifts (e.g. modal verbs), actions done for other purposes, … • It is designed for people acting Manage Incidental Interaction 22Intel® RealSense™ Hands-On Lab - Milan
  • 25. Gestures should have a clear cognitive association with the semantic functions they perform and the effects they achieve. Intuitiveness can be enforced by appropriate interface and feedbacks. The semantics of gestural patterns that belong to everyday life or common task should be as consistent as possible to their “conventional” meaning, but also take into account that intuitiveness is strongly associated with users’ cultural background, general knowledge, and experience. Semantic intuitiveness
  • 26. Gestural communication involves more muscles than keyboard interaction or speech. Gestural commands must therefore be concise and quick, and minimize user’s effort and physical stress. Two types of muscular stress are known: static, the effort required maintaining a posture for a fixed amount of time; dynamic, related to the effort required to move a portion of the body through a trajectory. Minimalize Fatigue
  • 27. It must be easy for the user to learn how to perform and remember gestures, minimizing the mental load of recalling movement trajectories and associated actions. The learning rate depends on tasks, user experience, skills, as well as the size of the gesture language (more gestures decrease the learnability rate). Favor ease of learning (Learnability) 1/2
  • 28. The gestures that are most natural, easy to learn and are immediately assimilated by the user are those that belong to everyday life, or involve the least physical effort. These gestures should be associated to the most frequent interactions. Complex gestures can be more expressive and give more control, but have a higher learnability burden. Hence there is clearly a tension between design requirements, among which a compromise must be made: naturalness of gestures, minimum size of the gesture language, expressiveness and completeness of the gesture language. Favor ease of learning (Learnability) 2/2
  • 29. Users can perform unintended gestures, i.e., movements that are not meant to communicate with the system they are interacting with. The “immersion syndrome” occurs if every movement is interpreted by the system, whether or not it was intended, and may determine interaction effects against the user’s will. Intentionality (Immersion Syndrome) 1/2
  • 30. The designer must identify well-defined means to detect the intention of the gestures, as distinguishing useful movements from unintentional ones is not easy. Body tension and non-relaxed posture of users can be used to make explicit the user intention to start interaction, issue a command, or confirm a choice. The tense period should be short to not generate fatigue. Intentionality (Immersion Syndrome) 2/2
  • 31. Appropriate feedback indicating the effects and correctness of the gesture performed is necessary for successful interaction, and to improve the user's confidence in the system. Not-self-revealing
  • 32. Intel RealSense Inspiration Projects 32Intel® RealSense™ Hands-On Lab - Milan
  • 34. 34 • The robot is built using Legos* Mindstorm • A standard Ultrabook is place on top • It acts as the brains • As well as the interface • Intel 3D Cameras are used to capture the environment & steer the robot around • It also allows for some facial & gesture recognition Rover the Self-Driving Car Intel® RealSense™ Hands-On Lab - Milan
  • 35. Space Between • Used both Gesture & Voice commands in this Puzzle Oriented Game about teaching a creature to survive in a virtual world • Used Unity* 3D as its game engine 35Intel® RealSense™ Hands-On Lab - Milan
  • 36. 3D Head Scanner • Rapidly (about a minute) does a full scan of the user’s head for avatar creation or other uses such as fashion & beauty applications • In the original app, users could change hair styles, skin complexion, etc. • Built as a Unity* project, it could be used by other applications & games for a wild array of different usage models 36Intel® RealSense™ Hands-On Lab - Milan
  • 37. Stargate SG1 Gunship V2 • An evolution of the award winning Perceptual Computing game • Controller-free, immersive UX • Two-handed, multi-axis camera and fire controls • Menu and in-game voice commands 37Intel® RealSense™ Hands-On Lab - Milan
  • 38. Redwall: The Warrior Reborn • Facial landmark mapping to character avatars • Fully controller free UI • Expression sensitive NPC interactions. 38Intel® RealSense™ Hands-On Lab - Milan
  • 40. WHAT IS THE EFFECTIVE RANGE FOR 3D FACIAL TRACKING? 40Intel® RealSense™ Hands-On Lab - Milan #RealSense @IntelRealSense