Human Action Recognition Based on Spacio-temporal features
Upcoming SlideShare
Loading in...5

Like this? Share it with your network


Human Action Recognition Based on Spacio-temporal features

Uploaded on


  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Be the first to comment
    Be the first to like this
No Downloads


Total Views
On Slideshare
From Embeds
Number of Embeds



Embeds 5 3 2

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

    No notes for slide


  • 1. Human Action Recognition Based on Spatio-temporal Features Nikhil Sawant and Dr. K.K. Biswas Dept. of CSE, Indian Institute of Technology, Delhi Third International Conference on Pattern Recognition and Machine Intelligence (PReMi’09)
  • 2. Human activity recognition Higher resolution Longer Time Scale Courtesy : Y. Ke, Fathi and Mori, Bobick and Davis, Schuldt et al, Leibe et al, Vaswani et al. Pose Estimation Event Detection Action Classification Tracking Activity Recognition
  • 3. Use Action recognition?
    • Video surveillance
    • Interactive environment
    • Video classification & indexing
    • Movie search
    • Assisted Care
    • Sports annotation
  • 4. Broad outline of our technique Action Class 1 Action Class 2 Action Class 3 Action Class n Motion analysis using Lucas –kanade technique Shape analysis using Viola-Jones feaures Combining motion and shape features over finite time interval …………… ...... Video with human actions Motions features Shape features Spatio-temporal features Learning features though AdaBoos
  • 5. Target Localization
    • Possible search space is xyt cube
    • Action needs to be localized in space and time
    • Target localization helps reducing search space
    • Background subtraction
    • ROI marked
    Original Video Silhouette Original Video with ROI marked
  • 6. Motion estimation
    • Make use of optical flows for motion estimation
    • Optical flow is the pattern of relative motion between the object/object feature points and the viewer/camera
    • We make use of Lucas – Kanade, two frame differential method, it comparatively yields robust and dense optical flows
  • 7. Noise Reduction
    • Noise removal by averaging
    • Optical flows with magnitude > C * O mean are ignored,
    • where C – constant [1.5 - 2],
    • O mean - mean of optical flow within ROI
  • 8.
    • Optical flows are aggregated near the motion
    • Need for representing optical flow in meaningful way
    • Fixed sized grid laid over the ROI
    Organizing optical flows
  • 9.
    • Magnitude and direction of Optical flows within each box b ij is averaged and assigned to its centre c ij
    • All optical flows have same weight
    Organizing optical flows (simple averaging)
  • 10.
    • Each optical flow given a weight
    • More the distance from the centre c ij less is the weight and vice-versa
    Organizing optical flows (weighted averaging)
  • 11.
    • Optical flows are arranged in structured mannered
    • Arranged optical flows are easier to analyze
    Organizing optical flows
  • 12. Shape discriptor
    • Shape gives information about the action
    • Viola-Jones box features used to get shape features
    • Shape information combined with motion information
  • 13. Spatio-temporal descriptor TLEN TSPAN
  • 14. Spatio-temporal descriptor
    • Shape and motion features combined over the span of time to form spatio-temporal features
  • 15. Learning with Adaboost
    • Adaboost is state of art learning algorithm
    • Linear decision stumps are used as weak hypothesis
    • Weak hypothesis combine to form a strong hypothesis
    • Strong hypothesis is weighted sum of weak hypothesis
    • Training and testing data is kept mutually exclusive
  • 16. Results
  • 17. Results (Weizman dataset)
  • 18. Thanks You