SlideShare a Scribd company logo
1 of 33
Download to read offline
Intel® RealSense™ Technology | Intel® Software
Kevin Arthur, Senior User Experience Researcher
Meghana Rao, Developer Evangelist
Augmented Reality with the
Intel® RealSense™ SDK and R200 Camera
User Experience and Development Best Practices
Intel® RealSense™ Technology | Intel® Software 2
Outline
Part 1, Kevin
•  Overview of R200 camera and tablet
augmented reality use cases
•  User experience guidelines highlights
Part 2, Meghana
•  SDK overview
•  Sample code and demos
Intel® RealSense™ Technology | Intel® Software 3
New R200 Depth Camera For Tablets,
Peripheral Dev Kit Available Now
Intel® RealSense™ Technology | Intel® Software 4
R200 View Volume and SDK Features
R200 with Intel® RealSense™ SDK
•  Scene Perception Module, enables
scene-aware AR
•  Camera tracking and localization
•  Mesh reconstruction
•  Other Modules
•  3D capture
•  Depth-enhanced photo and video
•  Measurement
•  Face detection and tracking
•  Speech (Windows SDK only)
Intel® RealSense™ Technology | Intel® Software 5
Use Cases for R200 Mixed and Augmented Reality
Gaming and Play Education and Training Visualization
Intel® RealSense™ Technology | Intel® Software 6
Video
Intel® RealSense™ Technology | Intel® Software 7
Video – ToyZ Game
Shows real-time scene
perception for collision
and occlusion (no pre-
scan)
Try it at the Intel booth
By Shachar Oz, Omek
Studio at Intel
Intel® RealSense™ Technology | Intel® Software 8
Video – Procedural Island
Shows scan as part of
“capture and play”
Illustrates procedural
shaders and set
dressing
By Eddy Ortega, Garrett
Stevens, Perceptual
Computing at Intel
Intel® RealSense™ Technology | Intel® Software
UX Guidelines for R200 Tablet AR
Designing real, usable apps for a mass market
Intel® RealSense™ Technology | Intel® Software 10
Lesson 1: Give People a Reason to Move, or They
Won’t
Tablet as window into a
virtual space vs. tablet as
fixed screen
Address with motion hints
Intel® RealSense™ Technology | Intel® Software 11
Motion Hints – Explicit
Registered with scene
Or registered with window
Intel® RealSense™ Technology | Intel® Software 12
Motion Hints – Implicit, Part of Experience
Lead the user with
content
Example: “Windy Day”
(Google Spotlight Stories)
Intel® RealSense™ Technology | Intel® Software 13
Lesson 2: But Let People Relax Too
Support both
Active Camera Mode
•  Tiring
Inactive Camera Mode
•  Less tiring
Intel® RealSense™ Technology | Intel® Software 14
Active-Camera and Inactive-Camera Modes
Active Camera Mode
•  Touch interaction less
comfortable, less precise
Inactive Camera Mode
•  Touch interaction more
comfortable, more precise
Make the main controls easy to reach with the
thumbs during Active Camera modes .
During Inactive Camera modes, placing
controls elsewhere is acceptable.
AVOID OKOKOK
Make the main controls easy to reach with the
thumbs during Active Camera modes .
During Inactive Camera modes, placing
controls elsewhere is acceptable.
AVOID OKOKOK
Touch Zones Touch Zones
Intel® RealSense™ Technology | Intel® Software 15
Two Styles of Mixed-Reality Games
Augmented Reality Capture and Play
Active Camera
Inactive Camera
Intel® RealSense™ Technology | Intel® Software 16
Lesson 3: Plan for the Scene
•  Consider size of play space, and use appropriate voxel resolution
Intel® RealSense™ Technology | Intel® Software 17
Game Design Considerations
•  What objects does the user need?
•  Level design has more unknowns
•  Enhance and transform everyday objects
in interesting ways
Intel® RealSense™ Technology | Intel® Software 18
Procedural Set Dressing
Intel® RealSense™ Technology | Intel® Software 19
Plan for the Scene
•  Understand the camera limitations.
Depth data is less accurate on
•  Very bright areas
•  Clear glass
•  Black surfaces
•  Give relevant feedback
•  Fail gracefully, don’t prevent play
Intel® RealSense™ Technology | Intel® Software 20
More UX Guidelines
software.intel.com/articles/realsense-ux-design-guidelines
Intel® RealSense™ Technology | Intel® Software
Developing Augmented Reality
applications with R200
Intel® RealSense™ Technology | Intel® Software 22
What is Scene Perception?
•  Provides Augmented Reality experiences through
•  Adding virtual content in real time to live camera feed
•  Handling occlusions of virtual objects with physical objects
•  Providing motion estimation and metric measurement
•  Intel® RealSense™ SDK library for Scene Perception
•  libPXCScenePerception
Intel® RealSense™ Technology | Intel® Software 23
Scene Perception in Action
Intel® RealSense™ Technology | Intel® Software 24
Augmented Reality in Action
Intel® RealSense™ Technology | Intel® Software 25
Metric Measurements in Action
Intel® RealSense™ Technology | Intel® Software 26
Requirements for Scene Perception
•  The scene being captured has to be static
•  Works within the bounds of a close object / cubicle space/ living room
•  You can track camera movement in 3D space along six degrees of freedom
-courtesy: http://en.wikipedia.org/wiki/Six_degrees_of_freedom
Intel® RealSense™ Technology | Intel® Software
Modalities you will learn about ( R200 )
•  Using Scene Perception for Augmented Reality
•  Configuring Scene Perception
•  Setting Initial Camera Pose
•  Setting Meshing Threshold
•  Setting Voxel Resolution
•  Starting Scene Quality
•  Checking Scene Quality
•  Enabling Scene Reconstruction
•  Tracking Data
•  Camera Pose
•  Volume Preview
•  Mesh Data
Intel® RealSense™ Technology | Intel® Software 28
Guidelines for using Scene Perception
•  Works best in the depth camera working range
•  The observed environment is assumed to be containing some visual texture
and/or having some 3D structure:
•  Tracking may be inaccurate when tracking reflective or monotonous surfaces, flat
walls, dark black areas or scene with limited geometric structure
•  Using appropriate VoxelResolution::XYZ_RESOLUTION settings:
•  VoxelResolution::HIGH_RESOLUTION for object size use cases
•  VoxelResolution::LOW_RESOLUTION for room sized environments
Intel® RealSense™ Technology | Intel® Software
Intel® RealSense™ Front-Facing 3D
Camera (F200) and Augmented
Reality Usages
Intel® RealSense™ Technology | Intel® Software 30
Augmented Reality apps can make use
of:
•  Contour Mode
•  Hand Data
•  Joint Data
•  Finger Data
•  Extremity points
•  Hand Openness
•  Usages
•  Nail Salon
•  Virtual Shopping – Jewelry
•  Games
AR Usages Using Hand Module
Intel® RealSense™ Technology | Intel® Software 31
Augmented Reality apps can make use
of:
•  Landmark
•  Pose detection
•  Eye Movement
•  Expression Detection
Usages:
•  Face Makeup
•  Virtual Shopping for glasses
•  Games/Entertainment
•  Plastic Surgery
AR Usages using Face Module
Intel® RealSense™ Technology | Intel® Software 32
Resources
software.intel.com/realsense
software.intel.com/articles/realsense-ux-design-guidelines
kevin.arthur@intel.com, @karthur
meghana.s.rao@intel.com, @RaoIntel
Intel® RealSense™ Technology | Intel® Software

More Related Content

What's hot

Intel® RealSense™ Technology: Code Walk-through Presented by Intel Software I...
Intel® RealSense™ Technology: Code Walk-through Presented by Intel Software I...Intel® RealSense™ Technology: Code Walk-through Presented by Intel Software I...
Intel® RealSense™ Technology: Code Walk-through Presented by Intel Software I...Intel® Software
 
Dev09 – la battaglia del touchless
Dev09 – la battaglia del touchlessDev09 – la battaglia del touchless
Dev09 – la battaglia del touchlessClemente Giorio
 
Writing applications using the Microsoft Kinect Sensor
Writing applications using the Microsoft Kinect SensorWriting applications using the Microsoft Kinect Sensor
Writing applications using the Microsoft Kinect Sensorphildenoncourt
 
Microsoft Kinect and Kinect SDK
Microsoft Kinect and Kinect SDKMicrosoft Kinect and Kinect SDK
Microsoft Kinect and Kinect SDKSiraj Memon
 
Introduction to Google Project Tango and Intel® RealSense™
Introduction to Google Project Tango and Intel® RealSense™Introduction to Google Project Tango and Intel® RealSense™
Introduction to Google Project Tango and Intel® RealSense™Francesca Tosi
 
Hololens: Primo Contatto - Marco Dal Pino - Codemotion Milan 2016
Hololens: Primo Contatto - Marco Dal Pino - Codemotion Milan 2016Hololens: Primo Contatto - Marco Dal Pino - Codemotion Milan 2016
Hololens: Primo Contatto - Marco Dal Pino - Codemotion Milan 2016Codemotion
 
Touchless touchscreen
Touchless touchscreenTouchless touchscreen
Touchless touchscreenNaga Dinesh
 
Microsoft Kinect in Healthcare
Microsoft Kinect in HealthcareMicrosoft Kinect in Healthcare
Microsoft Kinect in HealthcareGSW
 
Web cam sensing using sdk tool
Web cam sensing using sdk tool Web cam sensing using sdk tool
Web cam sensing using sdk tool eSAT Journals
 
Touchless technology
Touchless technologyTouchless technology
Touchless technologyInternet User
 
Touchless technology Seminar Presentation
Touchless technology Seminar PresentationTouchless technology Seminar Presentation
Touchless technology Seminar PresentationAparna Nk
 
Touchless Touchscreen
Touchless TouchscreenTouchless Touchscreen
Touchless TouchscreenTasnin Khan
 
The power of Kinect in 10 minutes
The power of Kinect in 10 minutesThe power of Kinect in 10 minutes
The power of Kinect in 10 minutesTom Kerkhove
 
Touchless interactivity is the new frontier
Touchless interactivity is the new frontierTouchless interactivity is the new frontier
Touchless interactivity is the new frontierLM3LABS
 
Touchless Touchscreen Technology
Touchless Touchscreen TechnologyTouchless Touchscreen Technology
Touchless Touchscreen TechnologySaurabh Tripathi
 

What's hot (20)

Intel® RealSense™ Technology: Code Walk-through Presented by Intel Software I...
Intel® RealSense™ Technology: Code Walk-through Presented by Intel Software I...Intel® RealSense™ Technology: Code Walk-through Presented by Intel Software I...
Intel® RealSense™ Technology: Code Walk-through Presented by Intel Software I...
 
Communitydays2015
Communitydays2015Communitydays2015
Communitydays2015
 
Dev09 – la battaglia del touchless
Dev09 – la battaglia del touchlessDev09 – la battaglia del touchless
Dev09 – la battaglia del touchless
 
Touch Less touch screen
Touch Less touch screenTouch Less touch screen
Touch Less touch screen
 
Writing applications using the Microsoft Kinect Sensor
Writing applications using the Microsoft Kinect SensorWriting applications using the Microsoft Kinect Sensor
Writing applications using the Microsoft Kinect Sensor
 
Microsoft Kinect and Kinect SDK
Microsoft Kinect and Kinect SDKMicrosoft Kinect and Kinect SDK
Microsoft Kinect and Kinect SDK
 
Introduction to Google Project Tango and Intel® RealSense™
Introduction to Google Project Tango and Intel® RealSense™Introduction to Google Project Tango and Intel® RealSense™
Introduction to Google Project Tango and Intel® RealSense™
 
Hololens: Primo Contatto - Marco Dal Pino - Codemotion Milan 2016
Hololens: Primo Contatto - Marco Dal Pino - Codemotion Milan 2016Hololens: Primo Contatto - Marco Dal Pino - Codemotion Milan 2016
Hololens: Primo Contatto - Marco Dal Pino - Codemotion Milan 2016
 
Touchless touchscreen
Touchless touchscreenTouchless touchscreen
Touchless touchscreen
 
Touchless Touch screen technology
Touchless Touch screen technologyTouchless Touch screen technology
Touchless Touch screen technology
 
Microsoft Kinect in Healthcare
Microsoft Kinect in HealthcareMicrosoft Kinect in Healthcare
Microsoft Kinect in Healthcare
 
Web cam sensing using sdk tool
Web cam sensing using sdk tool Web cam sensing using sdk tool
Web cam sensing using sdk tool
 
Communitydays2014
Communitydays2014Communitydays2014
Communitydays2014
 
Touchless technology
Touchless technologyTouchless technology
Touchless technology
 
Touchless technology Seminar Presentation
Touchless technology Seminar PresentationTouchless technology Seminar Presentation
Touchless technology Seminar Presentation
 
Kinect
KinectKinect
Kinect
 
Touchless Touchscreen
Touchless TouchscreenTouchless Touchscreen
Touchless Touchscreen
 
The power of Kinect in 10 minutes
The power of Kinect in 10 minutesThe power of Kinect in 10 minutes
The power of Kinect in 10 minutes
 
Touchless interactivity is the new frontier
Touchless interactivity is the new frontierTouchless interactivity is the new frontier
Touchless interactivity is the new frontier
 
Touchless Touchscreen Technology
Touchless Touchscreen TechnologyTouchless Touchscreen Technology
Touchless Touchscreen Technology
 

Similar to Augmented Reality with the Intel® RealSense™ SDK and R200 Camera

Building Applications with the Microsoft Kinect SDK
Building Applications with the Microsoft Kinect SDKBuilding Applications with the Microsoft Kinect SDK
Building Applications with the Microsoft Kinect SDKDataLeader.io
 
virtual reality Information-160422181930.pdf
virtual reality Information-160422181930.pdfvirtual reality Information-160422181930.pdf
virtual reality Information-160422181930.pdf21107117
 
Intel Real Sense, Diversity Meetup by Jamie Tanna
Intel Real Sense, Diversity Meetup by Jamie TannaIntel Real Sense, Diversity Meetup by Jamie Tanna
Intel Real Sense, Diversity Meetup by Jamie TannaBeMyApp
 
Kinect seminar 120919
Kinect seminar 120919Kinect seminar 120919
Kinect seminar 120919cs Kang
 
Philipp Nagele (Wikitude): Context Is for Kings: Putting Context in the Hands...
Philipp Nagele (Wikitude): Context Is for Kings: Putting Context in the Hands...Philipp Nagele (Wikitude): Context Is for Kings: Putting Context in the Hands...
Philipp Nagele (Wikitude): Context Is for Kings: Putting Context in the Hands...AugmentedWorldExpo
 
Synthetic environment
Synthetic environmentSynthetic environment
Synthetic environmentUllas Gupta
 
Nui e biometrics in windows 10
Nui e biometrics in windows 10Nui e biometrics in windows 10
Nui e biometrics in windows 10Marco D'Alessandro
 
JGJ48 : Intel Realsense - Firstman Marpaung
JGJ48 : Intel Realsense - Firstman MarpaungJGJ48 : Intel Realsense - Firstman Marpaung
JGJ48 : Intel Realsense - Firstman MarpaunggamelanYK
 
Virtual Reality
Virtual RealityVirtual Reality
Virtual RealityOmar Ghazi
 
2016 AR Summer School - Lecture 5
2016 AR Summer School - Lecture 52016 AR Summer School - Lecture 5
2016 AR Summer School - Lecture 5Mark Billinghurst
 
Kinect on Android Pandaboard
Kinect on Android PandaboardKinect on Android Pandaboard
Kinect on Android Pandaboardumituzun84
 
Building the Matrix: Your First VR App (SVCC 2016)
Building the Matrix: Your First VR App (SVCC 2016)Building the Matrix: Your First VR App (SVCC 2016)
Building the Matrix: Your First VR App (SVCC 2016)Liv Erickson
 
Kinect for Windows Quickstart Series
Kinect for Windows Quickstart SeriesKinect for Windows Quickstart Series
Kinect for Windows Quickstart SeriesRoberto Reto
 
'eyeSpace' platform for Orientation using Augmented Reality experience
'eyeSpace' platform for Orientation using Augmented Reality experience 'eyeSpace' platform for Orientation using Augmented Reality experience
'eyeSpace' platform for Orientation using Augmented Reality experience Benny Karov
 
Developing For Kinect For Windows
Developing For Kinect For WindowsDeveloping For Kinect For Windows
Developing For Kinect For WindowsPrashant Tiwari
 
NUX Presentation from TechMixer Birmingham 2011
NUX Presentation from TechMixer Birmingham 2011NUX Presentation from TechMixer Birmingham 2011
NUX Presentation from TechMixer Birmingham 2011Michael Heydt
 
1YGZf5i0MTTiWpZm216.pptx
1YGZf5i0MTTiWpZm216.pptx1YGZf5i0MTTiWpZm216.pptx
1YGZf5i0MTTiWpZm216.pptxVishuBaniyan
 

Similar to Augmented Reality with the Intel® RealSense™ SDK and R200 Camera (20)

Building Applications with the Microsoft Kinect SDK
Building Applications with the Microsoft Kinect SDKBuilding Applications with the Microsoft Kinect SDK
Building Applications with the Microsoft Kinect SDK
 
virtual reality Information-160422181930.pdf
virtual reality Information-160422181930.pdfvirtual reality Information-160422181930.pdf
virtual reality Information-160422181930.pdf
 
Intel Real Sense, Diversity Meetup by Jamie Tanna
Intel Real Sense, Diversity Meetup by Jamie TannaIntel Real Sense, Diversity Meetup by Jamie Tanna
Intel Real Sense, Diversity Meetup by Jamie Tanna
 
Virtual Reality
Virtual RealityVirtual Reality
Virtual Reality
 
Kinect seminar 120919
Kinect seminar 120919Kinect seminar 120919
Kinect seminar 120919
 
Philipp Nagele (Wikitude): Context Is for Kings: Putting Context in the Hands...
Philipp Nagele (Wikitude): Context Is for Kings: Putting Context in the Hands...Philipp Nagele (Wikitude): Context Is for Kings: Putting Context in the Hands...
Philipp Nagele (Wikitude): Context Is for Kings: Putting Context in the Hands...
 
Synthetic environment
Synthetic environmentSynthetic environment
Synthetic environment
 
Nui e biometrics in windows 10
Nui e biometrics in windows 10Nui e biometrics in windows 10
Nui e biometrics in windows 10
 
JGJ48 : Intel Realsense - Firstman Marpaung
JGJ48 : Intel Realsense - Firstman MarpaungJGJ48 : Intel Realsense - Firstman Marpaung
JGJ48 : Intel Realsense - Firstman Marpaung
 
Cybernetyx introduction
Cybernetyx introductionCybernetyx introduction
Cybernetyx introduction
 
Virtual Reality
Virtual RealityVirtual Reality
Virtual Reality
 
2016 AR Summer School - Lecture 5
2016 AR Summer School - Lecture 52016 AR Summer School - Lecture 5
2016 AR Summer School - Lecture 5
 
Kinect on Android Pandaboard
Kinect on Android PandaboardKinect on Android Pandaboard
Kinect on Android Pandaboard
 
Building the Matrix: Your First VR App (SVCC 2016)
Building the Matrix: Your First VR App (SVCC 2016)Building the Matrix: Your First VR App (SVCC 2016)
Building the Matrix: Your First VR App (SVCC 2016)
 
Kinect for Windows Quickstart Series
Kinect for Windows Quickstart SeriesKinect for Windows Quickstart Series
Kinect for Windows Quickstart Series
 
'eyeSpace' platform for Orientation using Augmented Reality experience
'eyeSpace' platform for Orientation using Augmented Reality experience 'eyeSpace' platform for Orientation using Augmented Reality experience
'eyeSpace' platform for Orientation using Augmented Reality experience
 
Kinect sensor
Kinect sensorKinect sensor
Kinect sensor
 
Developing For Kinect For Windows
Developing For Kinect For WindowsDeveloping For Kinect For Windows
Developing For Kinect For Windows
 
NUX Presentation from TechMixer Birmingham 2011
NUX Presentation from TechMixer Birmingham 2011NUX Presentation from TechMixer Birmingham 2011
NUX Presentation from TechMixer Birmingham 2011
 
1YGZf5i0MTTiWpZm216.pptx
1YGZf5i0MTTiWpZm216.pptx1YGZf5i0MTTiWpZm216.pptx
1YGZf5i0MTTiWpZm216.pptx
 

More from Intel® Software

AI for All: Biology is eating the world & AI is eating Biology
AI for All: Biology is eating the world & AI is eating Biology AI for All: Biology is eating the world & AI is eating Biology
AI for All: Biology is eating the world & AI is eating Biology Intel® Software
 
Python Data Science and Machine Learning at Scale with Intel and Anaconda
Python Data Science and Machine Learning at Scale with Intel and AnacondaPython Data Science and Machine Learning at Scale with Intel and Anaconda
Python Data Science and Machine Learning at Scale with Intel and AnacondaIntel® Software
 
Streamline End-to-End AI Pipelines with Intel, Databricks, and OmniSci
Streamline End-to-End AI Pipelines with Intel, Databricks, and OmniSciStreamline End-to-End AI Pipelines with Intel, Databricks, and OmniSci
Streamline End-to-End AI Pipelines with Intel, Databricks, and OmniSciIntel® Software
 
AI for good: Scaling AI in science, healthcare, and more.
AI for good: Scaling AI in science, healthcare, and more.AI for good: Scaling AI in science, healthcare, and more.
AI for good: Scaling AI in science, healthcare, and more.Intel® Software
 
Software AI Accelerators: The Next Frontier | Software for AI Optimization Su...
Software AI Accelerators: The Next Frontier | Software for AI Optimization Su...Software AI Accelerators: The Next Frontier | Software for AI Optimization Su...
Software AI Accelerators: The Next Frontier | Software for AI Optimization Su...Intel® Software
 
Advanced Techniques to Accelerate Model Tuning | Software for AI Optimization...
Advanced Techniques to Accelerate Model Tuning | Software for AI Optimization...Advanced Techniques to Accelerate Model Tuning | Software for AI Optimization...
Advanced Techniques to Accelerate Model Tuning | Software for AI Optimization...Intel® Software
 
Reducing Deep Learning Integration Costs and Maximizing Compute Efficiency| S...
Reducing Deep Learning Integration Costs and Maximizing Compute Efficiency| S...Reducing Deep Learning Integration Costs and Maximizing Compute Efficiency| S...
Reducing Deep Learning Integration Costs and Maximizing Compute Efficiency| S...Intel® Software
 
AWS & Intel Webinar Series - Accelerating AI Research
AWS & Intel Webinar Series - Accelerating AI ResearchAWS & Intel Webinar Series - Accelerating AI Research
AWS & Intel Webinar Series - Accelerating AI ResearchIntel® Software
 
Intel AIDC Houston Summit - Overview Slides
Intel AIDC Houston Summit - Overview SlidesIntel AIDC Houston Summit - Overview Slides
Intel AIDC Houston Summit - Overview SlidesIntel® Software
 
AIDC NY: BODO AI Presentation - 09.19.2019
AIDC NY: BODO AI Presentation - 09.19.2019AIDC NY: BODO AI Presentation - 09.19.2019
AIDC NY: BODO AI Presentation - 09.19.2019Intel® Software
 
AIDC NY: Applications of Intel AI by QuEST Global - 09.19.2019
AIDC NY: Applications of Intel AI by QuEST Global - 09.19.2019AIDC NY: Applications of Intel AI by QuEST Global - 09.19.2019
AIDC NY: Applications of Intel AI by QuEST Global - 09.19.2019Intel® Software
 
Advanced Single Instruction Multiple Data (SIMD) Programming with Intel® Impl...
Advanced Single Instruction Multiple Data (SIMD) Programming with Intel® Impl...Advanced Single Instruction Multiple Data (SIMD) Programming with Intel® Impl...
Advanced Single Instruction Multiple Data (SIMD) Programming with Intel® Impl...Intel® Software
 
Build a Deep Learning Video Analytics Framework | SIGGRAPH 2019 Technical Ses...
Build a Deep Learning Video Analytics Framework | SIGGRAPH 2019 Technical Ses...Build a Deep Learning Video Analytics Framework | SIGGRAPH 2019 Technical Ses...
Build a Deep Learning Video Analytics Framework | SIGGRAPH 2019 Technical Ses...Intel® Software
 
Bring Intelligent Motion Using Reinforcement Learning Engines | SIGGRAPH 2019...
Bring Intelligent Motion Using Reinforcement Learning Engines | SIGGRAPH 2019...Bring Intelligent Motion Using Reinforcement Learning Engines | SIGGRAPH 2019...
Bring Intelligent Motion Using Reinforcement Learning Engines | SIGGRAPH 2019...Intel® Software
 
RenderMan*: The Role of Open Shading Language (OSL) with Intel® Advanced Vect...
RenderMan*: The Role of Open Shading Language (OSL) with Intel® Advanced Vect...RenderMan*: The Role of Open Shading Language (OSL) with Intel® Advanced Vect...
RenderMan*: The Role of Open Shading Language (OSL) with Intel® Advanced Vect...Intel® Software
 
AIDC India - Intel Movidius / Open Vino Slides
AIDC India - Intel Movidius / Open Vino SlidesAIDC India - Intel Movidius / Open Vino Slides
AIDC India - Intel Movidius / Open Vino SlidesIntel® Software
 
AIDC India - AI Vision Slides
AIDC India - AI Vision SlidesAIDC India - AI Vision Slides
AIDC India - AI Vision SlidesIntel® Software
 
Enhance and Accelerate Your AI and Machine Learning Solution | SIGGRAPH 2019 ...
Enhance and Accelerate Your AI and Machine Learning Solution | SIGGRAPH 2019 ...Enhance and Accelerate Your AI and Machine Learning Solution | SIGGRAPH 2019 ...
Enhance and Accelerate Your AI and Machine Learning Solution | SIGGRAPH 2019 ...Intel® Software
 

More from Intel® Software (20)

AI for All: Biology is eating the world & AI is eating Biology
AI for All: Biology is eating the world & AI is eating Biology AI for All: Biology is eating the world & AI is eating Biology
AI for All: Biology is eating the world & AI is eating Biology
 
Python Data Science and Machine Learning at Scale with Intel and Anaconda
Python Data Science and Machine Learning at Scale with Intel and AnacondaPython Data Science and Machine Learning at Scale with Intel and Anaconda
Python Data Science and Machine Learning at Scale with Intel and Anaconda
 
Streamline End-to-End AI Pipelines with Intel, Databricks, and OmniSci
Streamline End-to-End AI Pipelines with Intel, Databricks, and OmniSciStreamline End-to-End AI Pipelines with Intel, Databricks, and OmniSci
Streamline End-to-End AI Pipelines with Intel, Databricks, and OmniSci
 
AI for good: Scaling AI in science, healthcare, and more.
AI for good: Scaling AI in science, healthcare, and more.AI for good: Scaling AI in science, healthcare, and more.
AI for good: Scaling AI in science, healthcare, and more.
 
Software AI Accelerators: The Next Frontier | Software for AI Optimization Su...
Software AI Accelerators: The Next Frontier | Software for AI Optimization Su...Software AI Accelerators: The Next Frontier | Software for AI Optimization Su...
Software AI Accelerators: The Next Frontier | Software for AI Optimization Su...
 
Advanced Techniques to Accelerate Model Tuning | Software for AI Optimization...
Advanced Techniques to Accelerate Model Tuning | Software for AI Optimization...Advanced Techniques to Accelerate Model Tuning | Software for AI Optimization...
Advanced Techniques to Accelerate Model Tuning | Software for AI Optimization...
 
Reducing Deep Learning Integration Costs and Maximizing Compute Efficiency| S...
Reducing Deep Learning Integration Costs and Maximizing Compute Efficiency| S...Reducing Deep Learning Integration Costs and Maximizing Compute Efficiency| S...
Reducing Deep Learning Integration Costs and Maximizing Compute Efficiency| S...
 
AWS & Intel Webinar Series - Accelerating AI Research
AWS & Intel Webinar Series - Accelerating AI ResearchAWS & Intel Webinar Series - Accelerating AI Research
AWS & Intel Webinar Series - Accelerating AI Research
 
Intel Developer Program
Intel Developer ProgramIntel Developer Program
Intel Developer Program
 
Intel AIDC Houston Summit - Overview Slides
Intel AIDC Houston Summit - Overview SlidesIntel AIDC Houston Summit - Overview Slides
Intel AIDC Houston Summit - Overview Slides
 
AIDC NY: BODO AI Presentation - 09.19.2019
AIDC NY: BODO AI Presentation - 09.19.2019AIDC NY: BODO AI Presentation - 09.19.2019
AIDC NY: BODO AI Presentation - 09.19.2019
 
AIDC NY: Applications of Intel AI by QuEST Global - 09.19.2019
AIDC NY: Applications of Intel AI by QuEST Global - 09.19.2019AIDC NY: Applications of Intel AI by QuEST Global - 09.19.2019
AIDC NY: Applications of Intel AI by QuEST Global - 09.19.2019
 
Advanced Single Instruction Multiple Data (SIMD) Programming with Intel® Impl...
Advanced Single Instruction Multiple Data (SIMD) Programming with Intel® Impl...Advanced Single Instruction Multiple Data (SIMD) Programming with Intel® Impl...
Advanced Single Instruction Multiple Data (SIMD) Programming with Intel® Impl...
 
Build a Deep Learning Video Analytics Framework | SIGGRAPH 2019 Technical Ses...
Build a Deep Learning Video Analytics Framework | SIGGRAPH 2019 Technical Ses...Build a Deep Learning Video Analytics Framework | SIGGRAPH 2019 Technical Ses...
Build a Deep Learning Video Analytics Framework | SIGGRAPH 2019 Technical Ses...
 
Bring Intelligent Motion Using Reinforcement Learning Engines | SIGGRAPH 2019...
Bring Intelligent Motion Using Reinforcement Learning Engines | SIGGRAPH 2019...Bring Intelligent Motion Using Reinforcement Learning Engines | SIGGRAPH 2019...
Bring Intelligent Motion Using Reinforcement Learning Engines | SIGGRAPH 2019...
 
RenderMan*: The Role of Open Shading Language (OSL) with Intel® Advanced Vect...
RenderMan*: The Role of Open Shading Language (OSL) with Intel® Advanced Vect...RenderMan*: The Role of Open Shading Language (OSL) with Intel® Advanced Vect...
RenderMan*: The Role of Open Shading Language (OSL) with Intel® Advanced Vect...
 
AIDC India - AI on IA
AIDC India  - AI on IAAIDC India  - AI on IA
AIDC India - AI on IA
 
AIDC India - Intel Movidius / Open Vino Slides
AIDC India - Intel Movidius / Open Vino SlidesAIDC India - Intel Movidius / Open Vino Slides
AIDC India - Intel Movidius / Open Vino Slides
 
AIDC India - AI Vision Slides
AIDC India - AI Vision SlidesAIDC India - AI Vision Slides
AIDC India - AI Vision Slides
 
Enhance and Accelerate Your AI and Machine Learning Solution | SIGGRAPH 2019 ...
Enhance and Accelerate Your AI and Machine Learning Solution | SIGGRAPH 2019 ...Enhance and Accelerate Your AI and Machine Learning Solution | SIGGRAPH 2019 ...
Enhance and Accelerate Your AI and Machine Learning Solution | SIGGRAPH 2019 ...
 

Recently uploaded

Powering Real-Time Decisions with Continuous Data Streams
Powering Real-Time Decisions with Continuous Data StreamsPowering Real-Time Decisions with Continuous Data Streams
Powering Real-Time Decisions with Continuous Data StreamsSafe Software
 
Precise and Complete Requirements? An Elusive Goal
Precise and Complete Requirements? An Elusive GoalPrecise and Complete Requirements? An Elusive Goal
Precise and Complete Requirements? An Elusive GoalLionel Briand
 
Machine Learning Software Engineering Patterns and Their Engineering
Machine Learning Software Engineering Patterns and Their EngineeringMachine Learning Software Engineering Patterns and Their Engineering
Machine Learning Software Engineering Patterns and Their EngineeringHironori Washizaki
 
Effectively Troubleshoot 9 Types of OutOfMemoryError
Effectively Troubleshoot 9 Types of OutOfMemoryErrorEffectively Troubleshoot 9 Types of OutOfMemoryError
Effectively Troubleshoot 9 Types of OutOfMemoryErrorTier1 app
 
Large Language Models for Test Case Evolution and Repair
Large Language Models for Test Case Evolution and RepairLarge Language Models for Test Case Evolution and Repair
Large Language Models for Test Case Evolution and RepairLionel Briand
 
Introduction to Firebase Workshop Slides
Introduction to Firebase Workshop SlidesIntroduction to Firebase Workshop Slides
Introduction to Firebase Workshop Slidesvaideheekore1
 
Patterns for automating API delivery. API conference
Patterns for automating API delivery. API conferencePatterns for automating API delivery. API conference
Patterns for automating API delivery. API conferencessuser9e7c64
 
Enhancing Supply Chain Visibility with Cargo Cloud Solutions.pdf
Enhancing Supply Chain Visibility with Cargo Cloud Solutions.pdfEnhancing Supply Chain Visibility with Cargo Cloud Solutions.pdf
Enhancing Supply Chain Visibility with Cargo Cloud Solutions.pdfRTS corp
 
GraphSummit Madrid - Product Vision and Roadmap - Luis Salvador Neo4j
GraphSummit Madrid - Product Vision and Roadmap - Luis Salvador Neo4jGraphSummit Madrid - Product Vision and Roadmap - Luis Salvador Neo4j
GraphSummit Madrid - Product Vision and Roadmap - Luis Salvador Neo4jNeo4j
 
2024-04-09 - From Complexity to Clarity - AWS Summit AMS.pdf
2024-04-09 - From Complexity to Clarity - AWS Summit AMS.pdf2024-04-09 - From Complexity to Clarity - AWS Summit AMS.pdf
2024-04-09 - From Complexity to Clarity - AWS Summit AMS.pdfAndrey Devyatkin
 
Understanding Flamingo - DeepMind's VLM Architecture
Understanding Flamingo - DeepMind's VLM ArchitectureUnderstanding Flamingo - DeepMind's VLM Architecture
Understanding Flamingo - DeepMind's VLM Architecturerahul_net
 
Strategies for using alternative queries to mitigate zero results
Strategies for using alternative queries to mitigate zero resultsStrategies for using alternative queries to mitigate zero results
Strategies for using alternative queries to mitigate zero resultsJean Silva
 
What’s New in VictoriaMetrics: Q1 2024 Updates
What’s New in VictoriaMetrics: Q1 2024 UpdatesWhat’s New in VictoriaMetrics: Q1 2024 Updates
What’s New in VictoriaMetrics: Q1 2024 UpdatesVictoriaMetrics
 
The Role of IoT and Sensor Technology in Cargo Cloud Solutions.pptx
The Role of IoT and Sensor Technology in Cargo Cloud Solutions.pptxThe Role of IoT and Sensor Technology in Cargo Cloud Solutions.pptx
The Role of IoT and Sensor Technology in Cargo Cloud Solutions.pptxRTS corp
 
eSoftTools IMAP Backup Software and migration tools
eSoftTools IMAP Backup Software and migration toolseSoftTools IMAP Backup Software and migration tools
eSoftTools IMAP Backup Software and migration toolsosttopstonverter
 
Comparing Linux OS Image Update Models - EOSS 2024.pdf
Comparing Linux OS Image Update Models - EOSS 2024.pdfComparing Linux OS Image Update Models - EOSS 2024.pdf
Comparing Linux OS Image Update Models - EOSS 2024.pdfDrew Moseley
 
Post Quantum Cryptography – The Impact on Identity
Post Quantum Cryptography – The Impact on IdentityPost Quantum Cryptography – The Impact on Identity
Post Quantum Cryptography – The Impact on Identityteam-WIBU
 
OpenChain Education Work Group Monthly Meeting - 2024-04-10 - Full Recording
OpenChain Education Work Group Monthly Meeting - 2024-04-10 - Full RecordingOpenChain Education Work Group Monthly Meeting - 2024-04-10 - Full Recording
OpenChain Education Work Group Monthly Meeting - 2024-04-10 - Full RecordingShane Coughlan
 
Amazon Bedrock in Action - presentation of the Bedrock's capabilities
Amazon Bedrock in Action - presentation of the Bedrock's capabilitiesAmazon Bedrock in Action - presentation of the Bedrock's capabilities
Amazon Bedrock in Action - presentation of the Bedrock's capabilitiesKrzysztofKkol1
 
Revolutionizing the Digital Transformation Office - Leveraging OnePlan’s AI a...
Revolutionizing the Digital Transformation Office - Leveraging OnePlan’s AI a...Revolutionizing the Digital Transformation Office - Leveraging OnePlan’s AI a...
Revolutionizing the Digital Transformation Office - Leveraging OnePlan’s AI a...OnePlan Solutions
 

Recently uploaded (20)

Powering Real-Time Decisions with Continuous Data Streams
Powering Real-Time Decisions with Continuous Data StreamsPowering Real-Time Decisions with Continuous Data Streams
Powering Real-Time Decisions with Continuous Data Streams
 
Precise and Complete Requirements? An Elusive Goal
Precise and Complete Requirements? An Elusive GoalPrecise and Complete Requirements? An Elusive Goal
Precise and Complete Requirements? An Elusive Goal
 
Machine Learning Software Engineering Patterns and Their Engineering
Machine Learning Software Engineering Patterns and Their EngineeringMachine Learning Software Engineering Patterns and Their Engineering
Machine Learning Software Engineering Patterns and Their Engineering
 
Effectively Troubleshoot 9 Types of OutOfMemoryError
Effectively Troubleshoot 9 Types of OutOfMemoryErrorEffectively Troubleshoot 9 Types of OutOfMemoryError
Effectively Troubleshoot 9 Types of OutOfMemoryError
 
Large Language Models for Test Case Evolution and Repair
Large Language Models for Test Case Evolution and RepairLarge Language Models for Test Case Evolution and Repair
Large Language Models for Test Case Evolution and Repair
 
Introduction to Firebase Workshop Slides
Introduction to Firebase Workshop SlidesIntroduction to Firebase Workshop Slides
Introduction to Firebase Workshop Slides
 
Patterns for automating API delivery. API conference
Patterns for automating API delivery. API conferencePatterns for automating API delivery. API conference
Patterns for automating API delivery. API conference
 
Enhancing Supply Chain Visibility with Cargo Cloud Solutions.pdf
Enhancing Supply Chain Visibility with Cargo Cloud Solutions.pdfEnhancing Supply Chain Visibility with Cargo Cloud Solutions.pdf
Enhancing Supply Chain Visibility with Cargo Cloud Solutions.pdf
 
GraphSummit Madrid - Product Vision and Roadmap - Luis Salvador Neo4j
GraphSummit Madrid - Product Vision and Roadmap - Luis Salvador Neo4jGraphSummit Madrid - Product Vision and Roadmap - Luis Salvador Neo4j
GraphSummit Madrid - Product Vision and Roadmap - Luis Salvador Neo4j
 
2024-04-09 - From Complexity to Clarity - AWS Summit AMS.pdf
2024-04-09 - From Complexity to Clarity - AWS Summit AMS.pdf2024-04-09 - From Complexity to Clarity - AWS Summit AMS.pdf
2024-04-09 - From Complexity to Clarity - AWS Summit AMS.pdf
 
Understanding Flamingo - DeepMind's VLM Architecture
Understanding Flamingo - DeepMind's VLM ArchitectureUnderstanding Flamingo - DeepMind's VLM Architecture
Understanding Flamingo - DeepMind's VLM Architecture
 
Strategies for using alternative queries to mitigate zero results
Strategies for using alternative queries to mitigate zero resultsStrategies for using alternative queries to mitigate zero results
Strategies for using alternative queries to mitigate zero results
 
What’s New in VictoriaMetrics: Q1 2024 Updates
What’s New in VictoriaMetrics: Q1 2024 UpdatesWhat’s New in VictoriaMetrics: Q1 2024 Updates
What’s New in VictoriaMetrics: Q1 2024 Updates
 
The Role of IoT and Sensor Technology in Cargo Cloud Solutions.pptx
The Role of IoT and Sensor Technology in Cargo Cloud Solutions.pptxThe Role of IoT and Sensor Technology in Cargo Cloud Solutions.pptx
The Role of IoT and Sensor Technology in Cargo Cloud Solutions.pptx
 
eSoftTools IMAP Backup Software and migration tools
eSoftTools IMAP Backup Software and migration toolseSoftTools IMAP Backup Software and migration tools
eSoftTools IMAP Backup Software and migration tools
 
Comparing Linux OS Image Update Models - EOSS 2024.pdf
Comparing Linux OS Image Update Models - EOSS 2024.pdfComparing Linux OS Image Update Models - EOSS 2024.pdf
Comparing Linux OS Image Update Models - EOSS 2024.pdf
 
Post Quantum Cryptography – The Impact on Identity
Post Quantum Cryptography – The Impact on IdentityPost Quantum Cryptography – The Impact on Identity
Post Quantum Cryptography – The Impact on Identity
 
OpenChain Education Work Group Monthly Meeting - 2024-04-10 - Full Recording
OpenChain Education Work Group Monthly Meeting - 2024-04-10 - Full RecordingOpenChain Education Work Group Monthly Meeting - 2024-04-10 - Full Recording
OpenChain Education Work Group Monthly Meeting - 2024-04-10 - Full Recording
 
Amazon Bedrock in Action - presentation of the Bedrock's capabilities
Amazon Bedrock in Action - presentation of the Bedrock's capabilitiesAmazon Bedrock in Action - presentation of the Bedrock's capabilities
Amazon Bedrock in Action - presentation of the Bedrock's capabilities
 
Revolutionizing the Digital Transformation Office - Leveraging OnePlan’s AI a...
Revolutionizing the Digital Transformation Office - Leveraging OnePlan’s AI a...Revolutionizing the Digital Transformation Office - Leveraging OnePlan’s AI a...
Revolutionizing the Digital Transformation Office - Leveraging OnePlan’s AI a...
 

Augmented Reality with the Intel® RealSense™ SDK and R200 Camera

  • 1. Intel® RealSense™ Technology | Intel® Software Kevin Arthur, Senior User Experience Researcher Meghana Rao, Developer Evangelist Augmented Reality with the Intel® RealSense™ SDK and R200 Camera User Experience and Development Best Practices
  • 2. Intel® RealSense™ Technology | Intel® Software 2 Outline Part 1, Kevin •  Overview of R200 camera and tablet augmented reality use cases •  User experience guidelines highlights Part 2, Meghana •  SDK overview •  Sample code and demos
  • 3. Intel® RealSense™ Technology | Intel® Software 3 New R200 Depth Camera For Tablets, Peripheral Dev Kit Available Now
  • 4. Intel® RealSense™ Technology | Intel® Software 4 R200 View Volume and SDK Features R200 with Intel® RealSense™ SDK •  Scene Perception Module, enables scene-aware AR •  Camera tracking and localization •  Mesh reconstruction •  Other Modules •  3D capture •  Depth-enhanced photo and video •  Measurement •  Face detection and tracking •  Speech (Windows SDK only)
  • 5. Intel® RealSense™ Technology | Intel® Software 5 Use Cases for R200 Mixed and Augmented Reality Gaming and Play Education and Training Visualization
  • 6. Intel® RealSense™ Technology | Intel® Software 6 Video
  • 7. Intel® RealSense™ Technology | Intel® Software 7 Video – ToyZ Game Shows real-time scene perception for collision and occlusion (no pre- scan) Try it at the Intel booth By Shachar Oz, Omek Studio at Intel
  • 8. Intel® RealSense™ Technology | Intel® Software 8 Video – Procedural Island Shows scan as part of “capture and play” Illustrates procedural shaders and set dressing By Eddy Ortega, Garrett Stevens, Perceptual Computing at Intel
  • 9. Intel® RealSense™ Technology | Intel® Software UX Guidelines for R200 Tablet AR Designing real, usable apps for a mass market
  • 10. Intel® RealSense™ Technology | Intel® Software 10 Lesson 1: Give People a Reason to Move, or They Won’t Tablet as window into a virtual space vs. tablet as fixed screen Address with motion hints
  • 11. Intel® RealSense™ Technology | Intel® Software 11 Motion Hints – Explicit Registered with scene Or registered with window
  • 12. Intel® RealSense™ Technology | Intel® Software 12 Motion Hints – Implicit, Part of Experience Lead the user with content Example: “Windy Day” (Google Spotlight Stories)
  • 13. Intel® RealSense™ Technology | Intel® Software 13 Lesson 2: But Let People Relax Too Support both Active Camera Mode •  Tiring Inactive Camera Mode •  Less tiring
  • 14. Intel® RealSense™ Technology | Intel® Software 14 Active-Camera and Inactive-Camera Modes Active Camera Mode •  Touch interaction less comfortable, less precise Inactive Camera Mode •  Touch interaction more comfortable, more precise Make the main controls easy to reach with the thumbs during Active Camera modes . During Inactive Camera modes, placing controls elsewhere is acceptable. AVOID OKOKOK Make the main controls easy to reach with the thumbs during Active Camera modes . During Inactive Camera modes, placing controls elsewhere is acceptable. AVOID OKOKOK Touch Zones Touch Zones
  • 15. Intel® RealSense™ Technology | Intel® Software 15 Two Styles of Mixed-Reality Games Augmented Reality Capture and Play Active Camera Inactive Camera
  • 16. Intel® RealSense™ Technology | Intel® Software 16 Lesson 3: Plan for the Scene •  Consider size of play space, and use appropriate voxel resolution
  • 17. Intel® RealSense™ Technology | Intel® Software 17 Game Design Considerations •  What objects does the user need? •  Level design has more unknowns •  Enhance and transform everyday objects in interesting ways
  • 18. Intel® RealSense™ Technology | Intel® Software 18 Procedural Set Dressing
  • 19. Intel® RealSense™ Technology | Intel® Software 19 Plan for the Scene •  Understand the camera limitations. Depth data is less accurate on •  Very bright areas •  Clear glass •  Black surfaces •  Give relevant feedback •  Fail gracefully, don’t prevent play
  • 20. Intel® RealSense™ Technology | Intel® Software 20 More UX Guidelines software.intel.com/articles/realsense-ux-design-guidelines
  • 21. Intel® RealSense™ Technology | Intel® Software Developing Augmented Reality applications with R200
  • 22. Intel® RealSense™ Technology | Intel® Software 22 What is Scene Perception? •  Provides Augmented Reality experiences through •  Adding virtual content in real time to live camera feed •  Handling occlusions of virtual objects with physical objects •  Providing motion estimation and metric measurement •  Intel® RealSense™ SDK library for Scene Perception •  libPXCScenePerception
  • 23. Intel® RealSense™ Technology | Intel® Software 23 Scene Perception in Action
  • 24. Intel® RealSense™ Technology | Intel® Software 24 Augmented Reality in Action
  • 25. Intel® RealSense™ Technology | Intel® Software 25 Metric Measurements in Action
  • 26. Intel® RealSense™ Technology | Intel® Software 26 Requirements for Scene Perception •  The scene being captured has to be static •  Works within the bounds of a close object / cubicle space/ living room •  You can track camera movement in 3D space along six degrees of freedom -courtesy: http://en.wikipedia.org/wiki/Six_degrees_of_freedom
  • 27. Intel® RealSense™ Technology | Intel® Software Modalities you will learn about ( R200 ) •  Using Scene Perception for Augmented Reality •  Configuring Scene Perception •  Setting Initial Camera Pose •  Setting Meshing Threshold •  Setting Voxel Resolution •  Starting Scene Quality •  Checking Scene Quality •  Enabling Scene Reconstruction •  Tracking Data •  Camera Pose •  Volume Preview •  Mesh Data
  • 28. Intel® RealSense™ Technology | Intel® Software 28 Guidelines for using Scene Perception •  Works best in the depth camera working range •  The observed environment is assumed to be containing some visual texture and/or having some 3D structure: •  Tracking may be inaccurate when tracking reflective or monotonous surfaces, flat walls, dark black areas or scene with limited geometric structure •  Using appropriate VoxelResolution::XYZ_RESOLUTION settings: •  VoxelResolution::HIGH_RESOLUTION for object size use cases •  VoxelResolution::LOW_RESOLUTION for room sized environments
  • 29. Intel® RealSense™ Technology | Intel® Software Intel® RealSense™ Front-Facing 3D Camera (F200) and Augmented Reality Usages
  • 30. Intel® RealSense™ Technology | Intel® Software 30 Augmented Reality apps can make use of: •  Contour Mode •  Hand Data •  Joint Data •  Finger Data •  Extremity points •  Hand Openness •  Usages •  Nail Salon •  Virtual Shopping – Jewelry •  Games AR Usages Using Hand Module
  • 31. Intel® RealSense™ Technology | Intel® Software 31 Augmented Reality apps can make use of: •  Landmark •  Pose detection •  Eye Movement •  Expression Detection Usages: •  Face Makeup •  Virtual Shopping for glasses •  Games/Entertainment •  Plastic Surgery AR Usages using Face Module
  • 32. Intel® RealSense™ Technology | Intel® Software 32 Resources software.intel.com/realsense software.intel.com/articles/realsense-ux-design-guidelines kevin.arthur@intel.com, @karthur meghana.s.rao@intel.com, @RaoIntel
  • 33. Intel® RealSense™ Technology | Intel® Software