Recommended
More Related Content
Viewers also liked
Viewers also liked (20)
Similar to Defense
Similar to Defense (20)
Defense
- 1. Human Activity Simulation Modeling in Smart Spaces Sirui (Quincy) Liu Ph.D. Dissertation Defense
- 2. 1. Review 1) Motivation 2) Activity Playback Modeling 2. Activity Visualization Modeling 1) Persim 3D System Overview 2) Action Modeling 3) Object Modeling 4) Sensor Modeling 5) Character Modeling 6) Validation 3. Conclusions Human Activity Simulation Modeling in Smart Spaces
- 3. 1. Review 1) Motivation 2) Activity Playback Modeling 2. Activity Visualization Modeling 1) Persim 3D System Overview 2) Action Modeling 3) Object Modeling 4) Sensor Modeling 5) Character Modeling 6) Validation 3. Conclusions Human Activity Simulation Modeling in Smart Spaces
- 4. What are Activities of Daily Living (ADLs)?
- 5. The things we normally do in daily living including any daily activity we perform for self-care such as… Eating Bathing Dressing Toileting
- 6. Why model and simulate activities of daily living?
- 7. Gator Tech Smart House
- 8. Difficulties in Conducting Experiments in Real Smart Spaces • Significant cost and elaborate groundwork • Recruiting participants to live in an intrusive environment • Recording all human activities over a long period of time • Hard to be modified to test new theories, sensors and applications • Regulatory limitations prohibit use of human subjects in all scenarios
- 9. Model and simulate human activities.
- 10. • Have control over attributes of activities • Reflect effects of activities on the smart space • Capable of producing realistic visualizations Requirements of Activity Simulation Model
- 12. Persim 3D Simulation Manager Visualization Engine Activity Playback Modeling Activity Playback Algorithms Activity Model Output Activity Visualizations Sensor Datasets
- 13. Action 1 Action 2 Action 3Action 1 Action 2 Activity = Action Sequence
- 14. Action 1 Action 1 Action 2 Action 3Action 2 Activity = Action Sequence
- 15. 1. Review 1) Motivation 2) Activity Playback Modeling 2. Activity Visualization Modeling 1) Persim 3D System Overview 2) Action Modeling 3) Object Modeling 4) Sensor Modeling 5) Character Modeling 6) Validation 3. Conclusions Human Activity Simulation Modeling in Smart Spaces
- 16. 1. Review 1) Motivation 2) Activity Playback Modeling 2. Activity Visualization Modeling 1) Persim 3D System Overview 2) Action Modeling 3) Object Modeling 4) Sensor Modeling 5) Character Modeling 6) Validation 3. Conclusions Human Activity Simulation Modeling in Smart Spaces
- 17. Persim 3D Simulation Configuration Actual Dataset Validate Through Similarity Comparison Synthesized Dataset
- 18. Persim 3D Simulation Configuration Actual Dataset Validate Through Similarity Comparison Visualization Visual Feedback Synthesized Dataset Validate Through User Study
- 23. Activity Simulation Character Model Action Model Activity ModelObject Model Sensor ModelVisualization Engine GUI
- 24. Activity Simulation Character Model Action Model Activity ModelObject Model Sensor ModelVisualization Engine GUI Character Profiling GUI State Space Context Simulation APIs Context Simulation
- 25. Activity Simulation Character Model Action Model Activity ModelObject Model Sensor ModelVisualization Engine GUI Character Profiling GUI State Space Context Simulation APIs Activity 1Activity 2Activity 3Activity 4Activity 5
- 26. Early Persim 3D
- 28. No. Type Activity 1 Normal Washing Face 2 Normal Washing Hands 3 Normal Drinking Water 4 Normal Combing Hair 5 Normal Shaving 6 Normal Sleeping 7 Normal Using Bathroom 8 Normal Leaving Home 9 Normal Using Cellphone 10 Normal Taking Medication 11 Normal Watching TV 12 Abnormal Falling
- 29. No. Action No. Action 1 Turn On Light 14 Pick Up 2 Turn Off Light 15 Put Down 3 Wash Hands 16 Comb Hair 4 Dry Hands 17 Flush Toilet 5 Wash Face 18 Open Door 6 Dry Face 19 Close Door 7 Take A Pill 20 Fall 8 Sit Down 21 Lie Down 9 Stand Up 22 Turn Off Lamp 10 Use Toilet 23 Sleep 11 Drink 24 Get Up 12 Use Cellphone 25 Use TV Remote 13 Shave 26 Watch TV
- 32. • Name • ID • Model • Transform: position + rotation • Character’s transform • In-hand transform • Status • Animation: door opening/closing
- 34. • Name • Object • Animation • Effect: changes object’s status • Importance: optional or necessary • Max occurrence • Prerequisite
- 36. • ID • Type • Attaching entity • Value • Sensing function
- 37. Contact Sensor
- 38. Motion Sensor
- 40. RFID Reader Function Distance(m) 0 1 2 3 4 Animation Frame 0 30 60 90 120 150 180 210 240 270 300 330 360 390 420 Distance Between RFID Reader and Tag Return Value
- 41. IR Sensor
- 42. Type Installation Use Contact Attached to doors, windows, drawers, etc. Detect if two objects are in contact. Motion Installed in floor, chairs, beds, etc. Detect motion of and around the object. IR Installed in faucets and toilets. Detect if there are people within range. RFID Attached to objects. Identify people and objects.
- 46. • Character Animator • Animations • Movement Control • RFID Reader • Navigation System • Action Processor • Physics Engine • Object Manager
- 47. Grounded Sit Down/ Stand Up Lie Down/ Get Up Pick Up/ Put Down Other Actions Actions While Sitting Bed Actions Use Object Character Animator
- 48. Animations Asset Store MayaMotion Capture
- 50. + Motion Capture
- 51. +
- 52. Ethan Pearl
- 54. Transitioning Movements Position A, Posture A Position B, Posture B
- 55. • Character’s body movement: walking, turning, etc. • Not action, but fill the gap between actions. Transitioning Movements
- 56. Next Action Change Object’s Status Perform Transitioning Movements Play Next Action’s Animation Update Sensor Value Activity Visualization Engine Action 2 Action 3 … Action n Action 1 Activity Playback Engine Action Queue Activity Simulation Module
- 57. Turn On Light Wash Hands Dry Hands Turn Off Light Washing Hands
- 58. Turn On Light Dry Hands Walk to Bathroom Light Light On Turn Body Walk to Sink Wash HandsTurn BodyWalk to TowelTurn Body Walk to Bathroom Light Turn Off Light Light Off Walk Out of Bathroom Action Transitioning Movement Action Effect Washing Hands
- 60. Physics Engine
- 62. Persim 3D System Data Structure Environment Configuration Objects Sensors Characters Actions Activities Contexts Simulation Log Sensor Readings Character’s Movement Character’s Activities Character’s Actions
- 63. Persim 3D Simulation Configuration Actual Dataset Visualization Visual Feedback Synthesized Dataset Validate Through Similarity Comparison Validate Through User Study
- 64. Dataset Similarity Analysis User Study
- 65. Simulation Data Real Datavs
- 66. • R = sensor data of one activity from the real dataset • S = sensor data of the same activity from the synthesized dataset • ri = ith sensor event in R • si = ith sensor event in S • l = max {|R|, |S|} • 𝜅 = number of common sensor events in R and S • 𝛼 = number of sensor events that didn’t occur in the counterpart • 𝜂 = number of sensor events in R and S combined Weighted Euclidean Distance
- 67. • m = number of sensor events in R • n = number of sensor events in S Dataset Similarity
- 68. Activity Average Max Min Occurrence Sleeping 1.00 1.00 1.00 11 Using Bathroom 0.98 1.00 0.83 11 Taking Medication 0.86 1.00 0.57 3 Watching TV 0.76 1.00 0.50 4 Leaving Home 1.00 1.00 1.00 10 Dataset Similarity
- 69. Online User Study • Amazon Mechanical Turk. • 7 activities, 3 videos each activity. • Users rated realism on scale of 1 (very poor) to 10 (very high). • 952 workers. Every video was rated by 100 workers. • Age ranged from 15 to 77 (avg = 33.96, SD = 10.85).
- 72. Demo
- 73. 1. Review 1) Motivation 2) Activity Playback Modeling 2. Activity Visualization Modeling 1) Persim 3D System Overview 2) Action Modeling 3) Object Modeling 4) Sensor Modeling 5) Character Modeling 6) Validation 3. Conclusions Human Activity Simulation Modeling in Smart Spaces
- 74. 1. Review 1) Motivation 2) Activity Playback Modeling 2. Activity Visualization Modeling 1) Persim 3D System Overview 2) Action Modeling 3) Object Modeling 4) Sensor Modeling 5) Character Modeling 6) Validation 3. Conclusions Human Activity Simulation Modeling in Smart Spaces
- 76. Future Work • Virtual environment • Character modeling • Animation • User interface • Context simulation integration