Frontiers ofHuman Activity Analysis                J. K. Aggarwal               Michael S. Ryoo                 Kris M. Ki...
Overview           2
Machine point of view Activities as videos   Activity = a particular set of videos    T                                 ...
Activity classification Simple task of identifying videos   Categorize given videos into their types.      Known, limit...
Activity classification Activity categorization   Input = a video segment containing 1 activity                         ...
Activity detection  Search for the particular time interval     <starting time, ending time>     Video segment containi...
Activity detection by classification Binary classifier                          Push/not-push                            ...
Recognition process Represent videos in terms of features   Captures properties of activity videos                      ...
Taxonomy  Approach based taxonomy      Recognition approaches can be categorized.                                      H...
Single layered vs. hierarchical Single layered approaches T               Feature              extraction Hierarchical a...
Taxonomy – single layered These approaches recognize actions directly from  a sequence of images.                        ...
Single layered approaches Action representation   Video volumes themselves   Features directly extracted from videos A...
Taxonomy – Hierarchical                                             Hierarchical approaches                       Statisti...
Hierarchical approaches Layered approaches   Activities in terms of sub-events.   Human interactions      Multiple age...
Hierarchical approaches Activities as semantic structures    Activity = a concatenation of its sub-events    Human-orie...
Upcoming SlideShare
Loading in …5
×

cvpr2011: human activity recognition - part 2: overview

1,053 views

Published on

Published in: Technology, Education
0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total views
1,053
On SlideShare
0
From Embeds
0
Number of Embeds
3
Actions
Shares
0
Downloads
59
Comments
0
Likes
0
Embeds 0
No embeds

No notes for slide

cvpr2011: human activity recognition - part 2: overview

  1. 1. Frontiers ofHuman Activity Analysis J. K. Aggarwal Michael S. Ryoo Kris M. Kitani
  2. 2. Overview 2
  3. 3. Machine point of view Activities as videos  Activity = a particular set of videos T Hugging Pushing T Punching Shaking hands video space (640*480*100 D) 3
  4. 4. Activity classification Simple task of identifying videos  Categorize given videos into their types.  Known, limited number of classes  Assumes that each video contains a single activity ? Push Punch Kick Shake Hug 4
  5. 5. Activity classification Activity categorization  Input = a video segment containing 1 activity Hugging Pushing ? Punching Shaking hands 5
  6. 6. Activity detection  Search for the particular time interval  <starting time, ending time>  Video segment containing the activity Hugging PushingInput: continuous video stream Punching Shaking hands 6
  7. 7. Activity detection by classification Binary classifier Push/not-push Yes, a push. No, not a push. Classifier Sliding window technique  Classify all possible time intervals Pushing 7
  8. 8. Recognition process Represent videos in terms of features  Captures properties of activity videos Running Recognize activities by comparing video representations Not running  Decision boundary 8
  9. 9. Taxonomy  Approach based taxonomy  Recognition approaches can be categorized. Human activity recognition Single-layered Hierarchical approaches approaches Space-time Sequential Statistical Syntactic Description approaches approaches -basedTrajecto- Volumes Local Exemplar State-based ries features -based Aggarwal and Ryoo, ACM CSUR 2011 9
  10. 10. Single layered vs. hierarchical Single layered approaches T Feature extraction Hierarchical approaches T … Stretching <1, 20> T Withdrawing <21, 30> … 10
  11. 11. Taxonomy – single layered These approaches recognize actions directly from a sequence of images. 11
  12. 12. Single layered approaches Action representation  Video volumes themselves  Features directly extracted from videos Action classification  Machine learning techniques  Support vector machines  Hidden Markov models 12
  13. 13. Taxonomy – Hierarchical Hierarchical approaches Statistical Syntactic Description-based approaches approaches approaches [Pinhanez and Bobick ’98]Human actions [Nguyen et al. ’05] [Gupta et al. ’09] [Intille and Bobick ’99] [Ivanov and Bobick ’00] [Vu et al. ’03]Human-Human [Oliver et al. ’02] interactions [Joo and Chellapha ’06] [Ghanem et al. ’04] [Ryoo and Aggarwal ’06, ’09a] [Shi et al. ’04]O [Moore and Essa ’02]O [Siskind ’01]OHuman-Object [Yu and Aggarwal ’06]O [Minnen et al. ’03]O [Nevatia et al. ’03, ’04]O interactions [Damen and Hogg ’09]O [Kitani et al. ’07]O [Ryoo and Aggarwal ’07]O [Cupillard et al. ’02]G [Gong and Xiang ’03]GGroup activities [Ryoo and Aggarwal ’08, ’10]G [Zhang et al.’06]G [Dai et al.’08]G 13
  14. 14. Hierarchical approaches Layered approaches  Activities in terms of sub-events.  Human interactions  Multiple agents Suitable for activity-level recognition 14
  15. 15. Hierarchical approaches Activities as semantic structures  Activity = a concatenation of its sub-events  Human-oriented: high-level  Hierarchically organized representations Hand shake = “two persons do shake-action (stretches, stays stretched, withdraw) simultaneously, while touching”. Fighting -> Punching : 0.3 this==Push_interactions(p1,p2) | Punching Fighting : 0.7 i=Stretch(p1’s arm) j=Stay_Stretched(p1’s arm) Punching -> stretch withdraw : 0.8 k =Touching(p1, p2) l =Depart(p2, p1) | stretch stay_withdrawn : 0.1 | stay_stretched withdraw : 0.1 15

×