2. Introduction
• Build a virtual human using Unity
• Gesture recognition with Kinect.
• Control the virtual human with gestures
• Enable user to interact with the virtual human
5. Gesture Recognition
• Kinect can only recognize locations of joints and bones, not
gestures
• Use relative locations of various joints to recognize gestures
• Left hand move up/down: Compare location of left wrist with
shoulder left, hip center:
• Right hand move up/down: Compare location of right wrist
with shoulder right, hip center:
6. Control Virtual Human
• Unity program and Kinect program communicate via a socket.
• Change Heart Rate: Move Left hand up or down
• Change Blood Pressure: Move Right hand up or down
7.
8. Doing CPR
• Move both hands in and out.
• Kinect sends locations of Left Wrist/Right Wrist and Shoulder
Center joint to the Unity program.
• Measure distance between Left Wrist/Right Wrist and
Shoulder Center joint.
• The Unit program know that user want to do CPR.
9. Virtual Human Animations
• Mathematically Driven Virtual Human
• Human reactions to Low(<60), normal(60-100), higher heart
rate(100-150 and BP>120) ,heart attack(>150) and CPR
procedure.
• Motion captured from real human by MS Kinect
• Heart beating animation
• CPR animation
10. CPR posture identification
• From MS kinect tracking data of human skeleton.
• Bend both arms and release them would be count as one
effective electric shock using defibrillator .
• Three times of shocks would make the patient back to live.
