Kinect virtual-learning (SIM U KIN Graduation Project)
•Download as PPTX, PDF•
1 like•799 views
SIM U KIN project is a virtual soft skills educator that simulates presentation and interviewing processes and gives feedback on the weakness points which includes body language, facial expression and speech mistakes then recommends the correct behavior.
Recommended
Recommended
More Related Content
Similar to Kinect virtual-learning (SIM U KIN Graduation Project)
Similar to Kinect virtual-learning (SIM U KIN Graduation Project) (20)
Recently uploaded
Recently uploaded (20)
Kinect virtual-learning (SIM U KIN Graduation Project)
- 1. Kinect Virtual Learning Team members: Mohamed Hesham Mahmoud Soliman Ahmed Amr Yousef Ahmed Ahmed Nasser Date: 5/7/2015
- 2. Agenda • Problem & Solution • Presentation Application • Interviewing Application • Pattern Recognition techniques • Speech Recognition • Demo • Questions ?
- 3. Problem • Why Soft Skills ? • Business & Education • Chance !! • Software !!
- 4. Existing Solutions • Expensive Courses • Online Materials
- 5. Idea & Solution • Simulate the presentation & interview Processes • train the presenter & interviewee on the correct behavior Body movements Speech recognition Facial Expression • Feedback of weakness points
- 6. What is Kinect !
- 7. Message Delivery • Body Language • Speech • Facial Expression
- 11. Body Joints
- 13. Dynamic Mistakes
- 17. Static Mistakes
- 19. Body Language
- 20. Body Language
- 21. Body Language
- 22. Body Language
- 35. Facial Techniques Face Joints Face Orientation
- 36. Face Gestures
- 37. Facial Techniques Face Joints Face Orientation
- 38. Face Gestures
- 39. Face Gestures
- 40. Face Gestures
- 41. Face Gestures
- 42. Face Gestures
- 43. Face Gestures
- 57. Rule-Based Classifiers • Conditions Threshold • 12 Joints with XYZ Coordinates
- 58. Hidden Markov Model Classifiers • Dataset 50 sample for each mistake • Models 9 states 200 Observations • Phases Learning Decoding Evaluate
- 59. 80 82 84 86 88 90 92 94 96 Rule-Based HMM Arm Leg Body center Classifier Arm Leg Center Type Rule-Based 92% 88% 90% Static HMM 95% 85% 93% Static – Dynamic Results and Statistics
- 60. Classifiers Conclusion • HMM is more accurate than Rule-based and support Dynamic states • Rule-based is complex to detect specific threshold for different bodies
- 61. Gesture Detection Flow • Not all features are relevant to all gestures. Each gestures has its own feature vector. • More than one Gesture can happen at the same time. We group related gestures together under respective limbs. • Gestures can be related to more than one limb We divide gestures into parts called “states”. Problems :
- 62. Gesture Detection Flow Righ t Arm Hand Over Hand Hand On Waist Hand In Pocket Left Arm Hand Over Hand Hand On Waist Hand In Pocket Righ t Leg Cross Leg Up Leg Left Leg Cross Leg Up Leg body Leaning left Leaning right
- 63. Gesture Detection Flow Flow : • Each Gesture has a certain condition on the detected states. • Body consists of 5 Limbs. • Each Limb has a most probable state. • Each state has a classifier object that receives the feature vector. • Each state has its own feature vector.
- 64. Gesture Detection Flow Gesture types : • Static Gestures : No movement involved Happens when any state from a group of states happen. Happens when all states from a group of states happen. • Dynamic Gestures : Requires the body to move Happens when a sequence of states happen in a short period of time.
- 66. Speech Recognition • Speech recognition process
- 67. Speech Recognition • Presentation application (Filler Words) Fillers words and phrases people use to cover verbal gaps— are word crutches. Presenters often use them out of fear. • The most common fillers are: So, And, All right, Okay, Like, Now, Well, You know, Right, Um and Uh.
- 69. First Phase "Put Question"
- 70. Second Phase "Paragraph Test"
- 72. Conclusion • HMM is more accurate than Rule-based Classifiers. • Kinect is the best device to use due to infrared feature. • Kinect V2 is better than Kinect V1 in Joint detection. • Kinect V2 has face Joints property over Kinect V1.
- 73. Future Work • Interviewing Enhancement • Try other classifiers seeking better accuracy • Provide the Grammar Builder with more alternatives
- 74. Sponsorship
- 76. Interview Demo