cvpr2011: human activity recognition - part 2: overview
Upcoming SlideShare
Loading in...5

Like this? Share it with your network

  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Be the first to comment
    Be the first to like this
No Downloads


Total Views
On Slideshare
From Embeds
Number of Embeds



Embeds 0

No embeds

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

    No notes for slide


  • 1. Frontiers ofHuman Activity Analysis J. K. Aggarwal Michael S. Ryoo Kris M. Kitani
  • 2. Overview 2
  • 3. Machine point of view Activities as videos  Activity = a particular set of videos T Hugging Pushing T Punching Shaking hands video space (640*480*100 D) 3
  • 4. Activity classification Simple task of identifying videos  Categorize given videos into their types.  Known, limited number of classes  Assumes that each video contains a single activity ? Push Punch Kick Shake Hug 4
  • 5. Activity classification Activity categorization  Input = a video segment containing 1 activity Hugging Pushing ? Punching Shaking hands 5
  • 6. Activity detection  Search for the particular time interval  <starting time, ending time>  Video segment containing the activity Hugging PushingInput: continuous video stream Punching Shaking hands 6
  • 7. Activity detection by classification Binary classifier Push/not-push Yes, a push. No, not a push. Classifier Sliding window technique  Classify all possible time intervals Pushing 7
  • 8. Recognition process Represent videos in terms of features  Captures properties of activity videos Running Recognize activities by comparing video representations Not running  Decision boundary 8
  • 9. Taxonomy  Approach based taxonomy  Recognition approaches can be categorized. Human activity recognition Single-layered Hierarchical approaches approaches Space-time Sequential Statistical Syntactic Description approaches approaches -basedTrajecto- Volumes Local Exemplar State-based ries features -based Aggarwal and Ryoo, ACM CSUR 2011 9
  • 10. Single layered vs. hierarchical Single layered approaches T Feature extraction Hierarchical approaches T … Stretching <1, 20> T Withdrawing <21, 30> … 10
  • 11. Taxonomy – single layered These approaches recognize actions directly from a sequence of images. 11
  • 12. Single layered approaches Action representation  Video volumes themselves  Features directly extracted from videos Action classification  Machine learning techniques  Support vector machines  Hidden Markov models 12
  • 13. Taxonomy – Hierarchical Hierarchical approaches Statistical Syntactic Description-based approaches approaches approaches [Pinhanez and Bobick ’98]Human actions [Nguyen et al. ’05] [Gupta et al. ’09] [Intille and Bobick ’99] [Ivanov and Bobick ’00] [Vu et al. ’03]Human-Human [Oliver et al. ’02] interactions [Joo and Chellapha ’06] [Ghanem et al. ’04] [Ryoo and Aggarwal ’06, ’09a] [Shi et al. ’04]O [Moore and Essa ’02]O [Siskind ’01]OHuman-Object [Yu and Aggarwal ’06]O [Minnen et al. ’03]O [Nevatia et al. ’03, ’04]O interactions [Damen and Hogg ’09]O [Kitani et al. ’07]O [Ryoo and Aggarwal ’07]O [Cupillard et al. ’02]G [Gong and Xiang ’03]GGroup activities [Ryoo and Aggarwal ’08, ’10]G [Zhang et al.’06]G [Dai et al.’08]G 13
  • 14. Hierarchical approaches Layered approaches  Activities in terms of sub-events.  Human interactions  Multiple agents Suitable for activity-level recognition 14
  • 15. Hierarchical approaches Activities as semantic structures  Activity = a concatenation of its sub-events  Human-oriented: high-level  Hierarchically organized representations Hand shake = “two persons do shake-action (stretches, stays stretched, withdraw) simultaneously, while touching”. Fighting -> Punching : 0.3 this==Push_interactions(p1,p2) | Punching Fighting : 0.7 i=Stretch(p1’s arm) j=Stay_Stretched(p1’s arm) Punching -> stretch withdraw : 0.8 k =Touching(p1, p2) l =Depart(p2, p1) | stretch stay_withdrawn : 0.1 | stay_stretched withdraw : 0.1 15