SlideShare a Scribd company logo
Controlling Virtual Reality using Kinect
Jerry Le
Chao Mei
Nesthor Perez
Introduction
• Build a virtual human using Unity
• Gesture recognition with Kinect.
• Control the virtual human with gestures
• Enable user to interact with the virtual human
Gesture Recognition
Gesture Recognition
Gesture Recognition
• Kinect can only recognize locations of joints and bones, not
gestures
• Use relative locations of various joints to recognize gestures
• Left hand move up/down: Compare location of left wrist with
shoulder left, hip center:
• Right hand move up/down: Compare location of right wrist
with shoulder right, hip center:
Control Virtual Human
• Unity program and Kinect program communicate via a socket.
• Change Heart Rate: Move Left hand up or down
• Change Blood Pressure: Move Right hand up or down
Doing CPR
• Move both hands in and out.
• Kinect sends locations of Left Wrist/Right Wrist and Shoulder
Center joint to the Unity program.
• Measure distance between Left Wrist/Right Wrist and
Shoulder Center joint.
• The Unit program know that user want to do CPR.
Virtual Human Animations
• Mathematically Driven Virtual Human
• Human reactions to Low(<60), normal(60-100), higher heart
rate(100-150 and BP>120) ,heart attack(>150) and CPR
procedure.
• Motion captured from real human by MS Kinect
• Heart beating animation
• CPR animation
CPR posture identification
• From MS kinect tracking data of human skeleton.
• Bend both arms and release them would be count as one
effective electric shock using defibrillator .
• Three times of shocks would make the patient back to live.
Questions or Comments
Cgfp2

More Related Content

Viewers also liked

Factsheet01 stranasppk
Factsheet01 stranasppkFactsheet01 stranasppk
Factsheet01 stranasppk
Agustanto Imam Suprayoghie
 
FinalProject
FinalProjectFinalProject
FinalProjectsat3003
 
How to create a market for the traditional crafts men group
How to create a market for the traditional crafts men groupHow to create a market for the traditional crafts men group
How to create a market for the traditional crafts men groupFrederik Tamariska
 
Opium drug in afghanistan
Opium drug in afghanistanOpium drug in afghanistan
Opium drug in afghanistankarolyn6
 
2012 연간보고서 내지 인쇄용 수정
2012 연간보고서 내지 인쇄용 수정2012 연간보고서 내지 인쇄용 수정
2012 연간보고서 내지 인쇄용 수정khrf99
 
บทเรียนที่ 1
บทเรียนที่ 1บทเรียนที่ 1
บทเรียนที่ 1
krukengpr03
 
บทเรียนที่ 1
บทเรียนที่ 1บทเรียนที่ 1
บทเรียนที่ 1
sripayom
 
Comunicação Android Arduino - JASI 2015
Comunicação Android Arduino - JASI 2015Comunicação Android Arduino - JASI 2015
Comunicação Android Arduino - JASI 2015
Rodrigo Reis Alves
 
Draft master plan desa pampang harapan
Draft master plan desa pampang harapanDraft master plan desa pampang harapan
Draft master plan desa pampang harapanFrederik Tamariska
 
Development woven products
Development woven  productsDevelopment woven  products
Development woven products
Frederik Tamariska
 
Microsoft котляров
Microsoft котляровMicrosoft котляров
Microsoft котляровIvan Kotlyarov
 

Viewers also liked (19)

Factsheet01 stranasppk
Factsheet01 stranasppkFactsheet01 stranasppk
Factsheet01 stranasppk
 
Empresa
EmpresaEmpresa
Empresa
 
FinalProject
FinalProjectFinalProject
FinalProject
 
Cgfp3
Cgfp3Cgfp3
Cgfp3
 
Micro hydro
Micro hydroMicro hydro
Micro hydro
 
How to create a market for the traditional crafts men group
How to create a market for the traditional crafts men groupHow to create a market for the traditional crafts men group
How to create a market for the traditional crafts men group
 
Rome presentation
Rome presentationRome presentation
Rome presentation
 
Cgfp
CgfpCgfp
Cgfp
 
Opium drug in afghanistan
Opium drug in afghanistanOpium drug in afghanistan
Opium drug in afghanistan
 
2012 연간보고서 내지 인쇄용 수정
2012 연간보고서 내지 인쇄용 수정2012 연간보고서 내지 인쇄용 수정
2012 연간보고서 내지 인쇄용 수정
 
บทเรียนที่ 1
บทเรียนที่ 1บทเรียนที่ 1
บทเรียนที่ 1
 
บทเรียนที่ 1
บทเรียนที่ 1บทเรียนที่ 1
บทเรียนที่ 1
 
Charité Entrepreneurship Summit
Charité Entrepreneurship SummitCharité Entrepreneurship Summit
Charité Entrepreneurship Summit
 
Comunicação Android Arduino - JASI 2015
Comunicação Android Arduino - JASI 2015Comunicação Android Arduino - JASI 2015
Comunicação Android Arduino - JASI 2015
 
Draft master plan desa pampang harapan
Draft master plan desa pampang harapanDraft master plan desa pampang harapan
Draft master plan desa pampang harapan
 
Development woven products
Development woven  productsDevelopment woven  products
Development woven products
 
Pertanian organik
Pertanian organikPertanian organik
Pertanian organik
 
Microsoft котляров
Microsoft котляровMicrosoft котляров
Microsoft котляров
 
Kelab komputer
Kelab komputerKelab komputer
Kelab komputer
 

Similar to Cgfp2

Animal killer
Animal killerAnimal killer
Animal killer
Coda Chang
 
Animal killer
Animal killerAnimal killer
Animal killer
Coda Chang
 
How body movement influences Virtual Reality analgesia?
How body movement influences Virtual Reality analgesia?How body movement influences Virtual Reality analgesia?
How body movement influences Virtual Reality analgesia?
Interactive Technologies and Games: Education, Health and Disability
 
Animal killer
Animal killerAnimal killer
Animal killer
Coda Chang
 
HaiXiu: Emotion Recognition from Movements
HaiXiu: Emotion Recognition from MovementsHaiXiu: Emotion Recognition from Movements
HaiXiu: Emotion Recognition from Movements
Debmalya Sinha
 
Computer vision towards an automatic recognition of communicative gestures
Computer vision   towards an automatic recognition of communicative gesturesComputer vision   towards an automatic recognition of communicative gestures
Computer vision towards an automatic recognition of communicative gestures
Donato Di Pierro
 
Haptic Retargeting For Virtual Reality
Haptic Retargeting For Virtual RealityHaptic Retargeting For Virtual Reality
Haptic Retargeting For Virtual Reality
Soheil Khodayari
 

Similar to Cgfp2 (7)

Animal killer
Animal killerAnimal killer
Animal killer
 
Animal killer
Animal killerAnimal killer
Animal killer
 
How body movement influences Virtual Reality analgesia?
How body movement influences Virtual Reality analgesia?How body movement influences Virtual Reality analgesia?
How body movement influences Virtual Reality analgesia?
 
Animal killer
Animal killerAnimal killer
Animal killer
 
HaiXiu: Emotion Recognition from Movements
HaiXiu: Emotion Recognition from MovementsHaiXiu: Emotion Recognition from Movements
HaiXiu: Emotion Recognition from Movements
 
Computer vision towards an automatic recognition of communicative gestures
Computer vision   towards an automatic recognition of communicative gesturesComputer vision   towards an automatic recognition of communicative gestures
Computer vision towards an automatic recognition of communicative gestures
 
Haptic Retargeting For Virtual Reality
Haptic Retargeting For Virtual RealityHaptic Retargeting For Virtual Reality
Haptic Retargeting For Virtual Reality
 

Recently uploaded

Software Delivery At the Speed of AI: Inflectra Invests In AI-Powered Quality
Software Delivery At the Speed of AI: Inflectra Invests In AI-Powered QualitySoftware Delivery At the Speed of AI: Inflectra Invests In AI-Powered Quality
Software Delivery At the Speed of AI: Inflectra Invests In AI-Powered Quality
Inflectra
 
GDG Cloud Southlake #33: Boule & Rebala: Effective AppSec in SDLC using Deplo...
GDG Cloud Southlake #33: Boule & Rebala: Effective AppSec in SDLC using Deplo...GDG Cloud Southlake #33: Boule & Rebala: Effective AppSec in SDLC using Deplo...
GDG Cloud Southlake #33: Boule & Rebala: Effective AppSec in SDLC using Deplo...
James Anderson
 
De-mystifying Zero to One: Design Informed Techniques for Greenfield Innovati...
De-mystifying Zero to One: Design Informed Techniques for Greenfield Innovati...De-mystifying Zero to One: Design Informed Techniques for Greenfield Innovati...
De-mystifying Zero to One: Design Informed Techniques for Greenfield Innovati...
Product School
 
Connector Corner: Automate dynamic content and events by pushing a button
Connector Corner: Automate dynamic content and events by pushing a buttonConnector Corner: Automate dynamic content and events by pushing a button
Connector Corner: Automate dynamic content and events by pushing a button
DianaGray10
 
Bits & Pixels using AI for Good.........
Bits & Pixels using AI for Good.........Bits & Pixels using AI for Good.........
Bits & Pixels using AI for Good.........
Alison B. Lowndes
 
PCI PIN Basics Webinar from the Controlcase Team
PCI PIN Basics Webinar from the Controlcase TeamPCI PIN Basics Webinar from the Controlcase Team
PCI PIN Basics Webinar from the Controlcase Team
ControlCase
 
Neuro-symbolic is not enough, we need neuro-*semantic*
Neuro-symbolic is not enough, we need neuro-*semantic*Neuro-symbolic is not enough, we need neuro-*semantic*
Neuro-symbolic is not enough, we need neuro-*semantic*
Frank van Harmelen
 
Unsubscribed: Combat Subscription Fatigue With a Membership Mentality by Head...
Unsubscribed: Combat Subscription Fatigue With a Membership Mentality by Head...Unsubscribed: Combat Subscription Fatigue With a Membership Mentality by Head...
Unsubscribed: Combat Subscription Fatigue With a Membership Mentality by Head...
Product School
 
Transcript: Selling digital books in 2024: Insights from industry leaders - T...
Transcript: Selling digital books in 2024: Insights from industry leaders - T...Transcript: Selling digital books in 2024: Insights from industry leaders - T...
Transcript: Selling digital books in 2024: Insights from industry leaders - T...
BookNet Canada
 
Elevating Tactical DDD Patterns Through Object Calisthenics
Elevating Tactical DDD Patterns Through Object CalisthenicsElevating Tactical DDD Patterns Through Object Calisthenics
Elevating Tactical DDD Patterns Through Object Calisthenics
Dorra BARTAGUIZ
 
Essentials of Automations: Optimizing FME Workflows with Parameters
Essentials of Automations: Optimizing FME Workflows with ParametersEssentials of Automations: Optimizing FME Workflows with Parameters
Essentials of Automations: Optimizing FME Workflows with Parameters
Safe Software
 
Mission to Decommission: Importance of Decommissioning Products to Increase E...
Mission to Decommission: Importance of Decommissioning Products to Increase E...Mission to Decommission: Importance of Decommissioning Products to Increase E...
Mission to Decommission: Importance of Decommissioning Products to Increase E...
Product School
 
State of ICS and IoT Cyber Threat Landscape Report 2024 preview
State of ICS and IoT Cyber Threat Landscape Report 2024 previewState of ICS and IoT Cyber Threat Landscape Report 2024 preview
State of ICS and IoT Cyber Threat Landscape Report 2024 preview
Prayukth K V
 
Kubernetes & AI - Beauty and the Beast !?! @KCD Istanbul 2024
Kubernetes & AI - Beauty and the Beast !?! @KCD Istanbul 2024Kubernetes & AI - Beauty and the Beast !?! @KCD Istanbul 2024
Kubernetes & AI - Beauty and the Beast !?! @KCD Istanbul 2024
Tobias Schneck
 
Leading Change strategies and insights for effective change management pdf 1.pdf
Leading Change strategies and insights for effective change management pdf 1.pdfLeading Change strategies and insights for effective change management pdf 1.pdf
Leading Change strategies and insights for effective change management pdf 1.pdf
OnBoard
 
DevOps and Testing slides at DASA Connect
DevOps and Testing slides at DASA ConnectDevOps and Testing slides at DASA Connect
DevOps and Testing slides at DASA Connect
Kari Kakkonen
 
FIDO Alliance Osaka Seminar: Passkeys at Amazon.pdf
FIDO Alliance Osaka Seminar: Passkeys at Amazon.pdfFIDO Alliance Osaka Seminar: Passkeys at Amazon.pdf
FIDO Alliance Osaka Seminar: Passkeys at Amazon.pdf
FIDO Alliance
 
Knowledge engineering: from people to machines and back
Knowledge engineering: from people to machines and backKnowledge engineering: from people to machines and back
Knowledge engineering: from people to machines and back
Elena Simperl
 
The Future of Platform Engineering
The Future of Platform EngineeringThe Future of Platform Engineering
The Future of Platform Engineering
Jemma Hussein Allen
 
GenAISummit 2024 May 28 Sri Ambati Keynote: AGI Belongs to The Community in O...
GenAISummit 2024 May 28 Sri Ambati Keynote: AGI Belongs to The Community in O...GenAISummit 2024 May 28 Sri Ambati Keynote: AGI Belongs to The Community in O...
GenAISummit 2024 May 28 Sri Ambati Keynote: AGI Belongs to The Community in O...
Sri Ambati
 

Recently uploaded (20)

Software Delivery At the Speed of AI: Inflectra Invests In AI-Powered Quality
Software Delivery At the Speed of AI: Inflectra Invests In AI-Powered QualitySoftware Delivery At the Speed of AI: Inflectra Invests In AI-Powered Quality
Software Delivery At the Speed of AI: Inflectra Invests In AI-Powered Quality
 
GDG Cloud Southlake #33: Boule & Rebala: Effective AppSec in SDLC using Deplo...
GDG Cloud Southlake #33: Boule & Rebala: Effective AppSec in SDLC using Deplo...GDG Cloud Southlake #33: Boule & Rebala: Effective AppSec in SDLC using Deplo...
GDG Cloud Southlake #33: Boule & Rebala: Effective AppSec in SDLC using Deplo...
 
De-mystifying Zero to One: Design Informed Techniques for Greenfield Innovati...
De-mystifying Zero to One: Design Informed Techniques for Greenfield Innovati...De-mystifying Zero to One: Design Informed Techniques for Greenfield Innovati...
De-mystifying Zero to One: Design Informed Techniques for Greenfield Innovati...
 
Connector Corner: Automate dynamic content and events by pushing a button
Connector Corner: Automate dynamic content and events by pushing a buttonConnector Corner: Automate dynamic content and events by pushing a button
Connector Corner: Automate dynamic content and events by pushing a button
 
Bits & Pixels using AI for Good.........
Bits & Pixels using AI for Good.........Bits & Pixels using AI for Good.........
Bits & Pixels using AI for Good.........
 
PCI PIN Basics Webinar from the Controlcase Team
PCI PIN Basics Webinar from the Controlcase TeamPCI PIN Basics Webinar from the Controlcase Team
PCI PIN Basics Webinar from the Controlcase Team
 
Neuro-symbolic is not enough, we need neuro-*semantic*
Neuro-symbolic is not enough, we need neuro-*semantic*Neuro-symbolic is not enough, we need neuro-*semantic*
Neuro-symbolic is not enough, we need neuro-*semantic*
 
Unsubscribed: Combat Subscription Fatigue With a Membership Mentality by Head...
Unsubscribed: Combat Subscription Fatigue With a Membership Mentality by Head...Unsubscribed: Combat Subscription Fatigue With a Membership Mentality by Head...
Unsubscribed: Combat Subscription Fatigue With a Membership Mentality by Head...
 
Transcript: Selling digital books in 2024: Insights from industry leaders - T...
Transcript: Selling digital books in 2024: Insights from industry leaders - T...Transcript: Selling digital books in 2024: Insights from industry leaders - T...
Transcript: Selling digital books in 2024: Insights from industry leaders - T...
 
Elevating Tactical DDD Patterns Through Object Calisthenics
Elevating Tactical DDD Patterns Through Object CalisthenicsElevating Tactical DDD Patterns Through Object Calisthenics
Elevating Tactical DDD Patterns Through Object Calisthenics
 
Essentials of Automations: Optimizing FME Workflows with Parameters
Essentials of Automations: Optimizing FME Workflows with ParametersEssentials of Automations: Optimizing FME Workflows with Parameters
Essentials of Automations: Optimizing FME Workflows with Parameters
 
Mission to Decommission: Importance of Decommissioning Products to Increase E...
Mission to Decommission: Importance of Decommissioning Products to Increase E...Mission to Decommission: Importance of Decommissioning Products to Increase E...
Mission to Decommission: Importance of Decommissioning Products to Increase E...
 
State of ICS and IoT Cyber Threat Landscape Report 2024 preview
State of ICS and IoT Cyber Threat Landscape Report 2024 previewState of ICS and IoT Cyber Threat Landscape Report 2024 preview
State of ICS and IoT Cyber Threat Landscape Report 2024 preview
 
Kubernetes & AI - Beauty and the Beast !?! @KCD Istanbul 2024
Kubernetes & AI - Beauty and the Beast !?! @KCD Istanbul 2024Kubernetes & AI - Beauty and the Beast !?! @KCD Istanbul 2024
Kubernetes & AI - Beauty and the Beast !?! @KCD Istanbul 2024
 
Leading Change strategies and insights for effective change management pdf 1.pdf
Leading Change strategies and insights for effective change management pdf 1.pdfLeading Change strategies and insights for effective change management pdf 1.pdf
Leading Change strategies and insights for effective change management pdf 1.pdf
 
DevOps and Testing slides at DASA Connect
DevOps and Testing slides at DASA ConnectDevOps and Testing slides at DASA Connect
DevOps and Testing slides at DASA Connect
 
FIDO Alliance Osaka Seminar: Passkeys at Amazon.pdf
FIDO Alliance Osaka Seminar: Passkeys at Amazon.pdfFIDO Alliance Osaka Seminar: Passkeys at Amazon.pdf
FIDO Alliance Osaka Seminar: Passkeys at Amazon.pdf
 
Knowledge engineering: from people to machines and back
Knowledge engineering: from people to machines and backKnowledge engineering: from people to machines and back
Knowledge engineering: from people to machines and back
 
The Future of Platform Engineering
The Future of Platform EngineeringThe Future of Platform Engineering
The Future of Platform Engineering
 
GenAISummit 2024 May 28 Sri Ambati Keynote: AGI Belongs to The Community in O...
GenAISummit 2024 May 28 Sri Ambati Keynote: AGI Belongs to The Community in O...GenAISummit 2024 May 28 Sri Ambati Keynote: AGI Belongs to The Community in O...
GenAISummit 2024 May 28 Sri Ambati Keynote: AGI Belongs to The Community in O...
 

Cgfp2

  • 1. Controlling Virtual Reality using Kinect Jerry Le Chao Mei Nesthor Perez
  • 2. Introduction • Build a virtual human using Unity • Gesture recognition with Kinect. • Control the virtual human with gestures • Enable user to interact with the virtual human
  • 5. Gesture Recognition • Kinect can only recognize locations of joints and bones, not gestures • Use relative locations of various joints to recognize gestures • Left hand move up/down: Compare location of left wrist with shoulder left, hip center: • Right hand move up/down: Compare location of right wrist with shoulder right, hip center:
  • 6. Control Virtual Human • Unity program and Kinect program communicate via a socket. • Change Heart Rate: Move Left hand up or down • Change Blood Pressure: Move Right hand up or down
  • 7.
  • 8. Doing CPR • Move both hands in and out. • Kinect sends locations of Left Wrist/Right Wrist and Shoulder Center joint to the Unity program. • Measure distance between Left Wrist/Right Wrist and Shoulder Center joint. • The Unit program know that user want to do CPR.
  • 9. Virtual Human Animations • Mathematically Driven Virtual Human • Human reactions to Low(<60), normal(60-100), higher heart rate(100-150 and BP>120) ,heart attack(>150) and CPR procedure. • Motion captured from real human by MS Kinect • Heart beating animation • CPR animation
  • 10. CPR posture identification • From MS kinect tracking data of human skeleton. • Bend both arms and release them would be count as one effective electric shock using defibrillator . • Three times of shocks would make the patient back to live.