Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Extended Co-occurrence HOG with Dense Trajectories for Fine-grained Activity Recognition- ACCV2014 paper

1,613 views

Published on

In this paper we propose a novel feature descriptor Extended Co-occurrence HOG (ECoHOG) and integrate it with dense point trajectories demonstrating its usefulness in fine grained activity recognition. This feature is inspired by original Co-occurrence HOG (CoHOG) that is based on histograms of occurrences of pairs of image gradients in the image. Instead relying only on pure histograms we introduce a sum of gradient magnitudes of co-occurring pairs of image gradients in the image. This results in giving the importance to the object boundaries and straightening the difference between the moving foreground and static background. We also couple ECoHOG with dense point trajectories extracted using optical flow from video sequences and demonstrate that they are extremely well suited for fine grained activity recognition. Using our feature we outperform state of the art methods in this task and provide extensive quantitative evaluation.

Published in: Engineering
  • Be the first to comment

  • Be the first to like this

Extended Co-occurrence HOG with Dense Trajectories for Fine-grained Activity Recognition- ACCV2014 paper

  1. 1. ACCV 2014 accepted paper. Extended Co-occurrence HOG with Dense Trajectories for Fine-grained Activity Recognition H. Kataoka, K. Hashimoto, K. Iwata, Y. Satoh, N. Navab, S. Ilic, Y. Aoki Now available here! http://www.hirokatsukataoka.net/
  2. 2. Fine-grained Activity Recognition h"ps://sites.google.com/site/cvprfgvctwo/home – The visual distinctions between similar categories are often quite subtle – It’s difficult to address with general-purpose object recognition machinery Pose-based approach Holistic appraoch Fine-grained recognition [1] M. Rohrbach+, “A Database for Fine Grained Activity Detection of Cooking Activities”, in CVPR2012.
  3. 3. Dense Trajectories[2] Dense sampling and feature description on tracks – Pyramidal image division, and dense tracks with optical flow – Feature description (HOG, HOF, MBHx, MBHy, Traj.) – Visual words representation [2] H. Wang+, “Dense Trajectories and Motion Boundary Descriptors for Action Recognition”, in IJCV2013.
  4. 4. Motivation Improved Co-occurrence feature in Dense Trajectories – Contribution • Detailed feature description with co-occurrence feature • Effective feature descriptor in fine-grained recognition – Approach • Co-occurrence HOG[3] with edge-magnitude (to enhance edge-boundary) • Dimension reduction with PCA • MPII cooking activities dataset & INRIA surgery dataset [3] T. Watanabe+, “Co-occurrence histograms of oriented gradients for pedestrian detection”, in PSIVT2009.
  5. 5. Co-occurrence feature:CoHOG&ECoHOG “Edge-pair counting” & “Edge-magnitude accumulation(proposal)” CoHOG: edge-pair counting to corresponding histogram position Extended CoHOG(ECoHOG): edge-magnitude accumulation – PCA dim. reduction: 103 - 104 dims into 101-102 ,easy to divide in feature space
  6. 6. Parameter decision “# of PCA dim” & “size of edge extraction window” (a) PCA [dimensions] – 5, 10, 20, 50, 100, 200 (b) PCA [dimensions] – 50, 60, 70, 80, 90, 100 (c) Size of edge extraction window [pixels] – 3x3, 5x5, 7x7, 9x9, 11x11 – PCA dim: 70 dimensions => consider “contribution ratio” & “feature size” – Window size: 5x5 pixels => pixel similarity
  7. 7. Top 50 frequently used features – Neighbor pixels are used in classification
  8. 8. Experiment on INRIA surgery & MPII dataset Approach Accuracy (%) HOG + Tracking 40.16 Original DT (HOG, HOF, MBH, Trajectory) 93.58 CoHOG in DT 81.05 ECoHOG in DT 96.36 All model (ECoHOG, HOF, MBH, Trajectory) 97.31 Approach Accuracy (%) HOG + Tracking 18.2 Original DT (HOG, HOF, MBH, Trajectory) 44.2 CoHOG in DT 46.2 ECoHOG in DT 46.6 All model (ECoHOG, HOF, MBH, Trajectory) 49.1 INRIA surgery dataset MPII cooking activities dataset
  9. 9. Consideration • Improved co-occurrence feature into dense traj. – Dense sampling and detailed description in human activity – To grab small feature changing • Parameter optimization – PCA dim: 70 dimensions => consider “contribution ratio” & “feature size” – Window size: 5x5 pixels => pixel similarity

×