Your SlideShare is downloading. ×
0
Fusion of Skeletal and Silhouette-based Features for Human Action Recognition with RGB-D Devices
Fusion of Skeletal and Silhouette-based Features for Human Action Recognition with RGB-D Devices
Fusion of Skeletal and Silhouette-based Features for Human Action Recognition with RGB-D Devices
Fusion of Skeletal and Silhouette-based Features for Human Action Recognition with RGB-D Devices
Fusion of Skeletal and Silhouette-based Features for Human Action Recognition with RGB-D Devices
Fusion of Skeletal and Silhouette-based Features for Human Action Recognition with RGB-D Devices
Fusion of Skeletal and Silhouette-based Features for Human Action Recognition with RGB-D Devices
Fusion of Skeletal and Silhouette-based Features for Human Action Recognition with RGB-D Devices
Fusion of Skeletal and Silhouette-based Features for Human Action Recognition with RGB-D Devices
Fusion of Skeletal and Silhouette-based Features for Human Action Recognition with RGB-D Devices
Fusion of Skeletal and Silhouette-based Features for Human Action Recognition with RGB-D Devices
Fusion of Skeletal and Silhouette-based Features for Human Action Recognition with RGB-D Devices
Fusion of Skeletal and Silhouette-based Features for Human Action Recognition with RGB-D Devices
Fusion of Skeletal and Silhouette-based Features for Human Action Recognition with RGB-D Devices
Fusion of Skeletal and Silhouette-based Features for Human Action Recognition with RGB-D Devices
Fusion of Skeletal and Silhouette-based Features for Human Action Recognition with RGB-D Devices
Fusion of Skeletal and Silhouette-based Features for Human Action Recognition with RGB-D Devices
Upcoming SlideShare
Loading in...5
×

Thanks for flagging this SlideShare!

Oops! An error has occurred.

×
Saving this for later? Get the SlideShare app to save on your phone or tablet. Read anywhere, anytime – even offline.
Text the download link to your phone
Standard text messaging rates apply

Fusion of Skeletal and Silhouette-based Features for Human Action Recognition with RGB-D Devices

985

Published on

Paper presented at the 3rd Workshop on Consumer Depth Cameras for Computer Vision (CDC4CV), ICCV 2013 Workshop, Sydney (Australia), 2013

Paper presented at the 3rd Workshop on Consumer Depth Cameras for Computer Vision (CDC4CV), ICCV 2013 Workshop, Sydney (Australia), 2013

Published in: Technology, Business
0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total Views
985
On Slideshare
0
From Embeds
0
Number of Embeds
1
Actions
Shares
0
Downloads
10
Comments
0
Likes
0
Embeds 0
No embeds

Report content
Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
No notes for slide

Transcript

  • 1. 3rd IEEE Workshop on Consumer Depth Cameras for Computer Vision (CDC4CV) Sydney, December 2, 2013 ALEXANDROS A. CHAARAOUI JOSÉ R. PADILLA-LÓPEZ FRANCISCO FLÓREZ-REVUELTA
  • 2. 1. Introduction © A.A. Chaaraoui, J.R. Padilla-López and F. Flórez-Revuelta (CDC4CV’13) 2  Motivation:  Use of both skeleton and silhouette in previous works  Problems with skeleton: lack of precision or noisy caused by occlusion caused by body parts or objects Pick-up and Throw
  • 3. 1. Introduction © A.A. Chaaraoui, J.R. Padilla-López and F. Flórez-Revuelta (CDC4CV’13) 3  Motivation:  Use of both skeleton and silhouette in previous works  Problems with silhouettes: the only available viewpoint is unfavourable for recognition Tennis Serve Forward Punch Hammer
  • 4. 1. Introduction © A.A. Chaaraoui, J.R. Padilla-López and F. Flórez-Revuelta (CDC4CV’13) 4  Solution:  Fusing different features that complement each other: skeleton, RGB colour, silhouette (2D), volume (3D)…  In this work, we fuse skeleton and silhouette
  • 5. 2. Fusion of skeleton and silhouette © A.A. Chaaraoui, J.R. Padilla-López and F. Flórez-Revuelta (CDC4CV’13) 5 Concatenation of skeleton and silhouette features  Skeleton:  Silhouette:  3D coordinates of the  joints 20 3  1 2 4 8 9 7 5 6 10 12 11 13 14 15 16 17 19 18 Radial summary
  • 6. 3. Classification method based on a bag of key poses © A.A. Chaaraoui, J.R. Padilla-López and F. Flórez-Revuelta (CDC4CV’13) 6 [1] A.A. Chaaraoui, P. Climent-Pérez and F. Flórez-Revuelta. Silhouette-based human action recognition using sequences of key poses. Pattern Recognition Letters, 34(15):1799-1807,
  • 7. 3. Classification method based on a bag of key poses © A.A. Chaaraoui, J.R. Padilla-López and F. Flórez-Revuelta (CDC4CV’13) 7 [1] A.A. Chaaraoui, P. Climent-Pérez and F. Flórez-Revuelta. Silhouette-based human action recognition using sequences of key poses. Pattern Recognition Letters, 34(15):1799-
  • 8. 3. Classification method based on a bag of key poses © A.A. Chaaraoui, J.R. Padilla-López and F. Flórez-Revuelta (CDC4CV’13) 8  Sequence recognition Transform a sequence into a sequences of key poses using the bag of key poses  Sequence matching using dynamic time warping  [1] A.A. Chaaraoui, P. Climent-Pérez and F. Flórez-Revuelta. Silhouette-based human action recognition using sequences of key poses. Pattern Recognition Letters, 34(15):1799-1807,
  • 9. © A.A. Chaaraoui, J.R. Padilla-López and F. Flórez-Revuelta (CDC4CV’13) 4. Experimentation 9  Evaluation with the MSR Action3D dataset
  • 10. 4. Experimentation © A.A. Chaaraoui, J.R. Padilla-López and F. Flórez-Revuelta (CDC4CV’13) 10  Cross-subject validation as in [2]: Training: actors 1, 3, 5, 7 and 9  Testing: actors 2, 4, 6, 8 and 10  [2] W. Li, Z. Zhang, and Z. Liu. Action recognition based on a bag of 3D points. In 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition Workshops, pp. 9-14, 2010.
  • 11. 4. Experimentation © A.A. Chaaraoui, J.R. Padilla-López and F. Flórez-Revuelta (CDC4CV’13) 11  Cross-subject validation as in [2]: [2] W. Li, Z. Zhang, and Z. Liu. Action recognition based on a bag of 3D points. In 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition Workshops, pp. 9-14, 2010.
  • 12. 4. Experimentation © A.A. Chaaraoui, J.R. Padilla-López and F. Flórez-Revuelta (CDC4CV’13) 12  Confusion matrices for AS1: a02 a03 a05 a06 a10 a13 a18 a20 a02 0,92 0,08 a03 1,00 a05 0,91 0,09 a06 0,09 0,73 0,18 a10 1,00 a13 1,00 a18 1,00 a20 0,14 0,07 0,29 0,50 a02 a03 a05 a06 a10 a13 a18 a20 a02 0,67 0,25 0,08 a03 0,58 0,42 a05 0,18 0,73 0,09 a06 0,18 0,82 a10 1,00 a13 0,07 0,93 a18 0,33 0,20 0,07 0,40 a20 0,07 0,14 0,07 0,14 0,57 Skeleton Silhouette a02 a03 a05 a06 a10 a13 a18 a20 a02 1,00 a03 1,00 a05 0,09 0,91 a06 0,18 0,73 0,09 a10 1,00 a13 1,00 a18 1,00 a20 0,29 0,71 Fusion
  • 13. © A.A. Chaaraoui, J.R. Padilla-López and F. Flórez-Revuelta (CDC4CV’13) 4. Experimentation 13  Leave-one-actor-out:
  • 14. 5. Conclusions and future work © A.A. Chaaraoui, J.R. Padilla-López and F. Flórez-Revuelta (CDC4CV’13) 14      Straightforward fusion of skeleton and silhouette Improvement in the recognition rate Include also side and top projected silhouettes Select the weight for each feature vector Feature subset selection
  • 15. 5. Conclusions and future work © A.A. Chaaraoui, J.R. Padilla-López and F. Flórez-Revuelta (CDC4CV’13) 15  We have already applied the approach in [3] for feature selection to the fusion of skeleton and silhouette Cross-Subject LOAO [3] A.A. Chaaraoui, J.R. Padilla-López, P. Climent-Pérez, and F. Flórez-Revuelta. Evolutionary joint selection to improve human action recognition with RGB-D devices. Expert Systems with Applications, 41(3):786-794,2014.
  • 16. 5. Conclusions and future work © A.A. Chaaraoui, J.R. Padilla-López and F. Flórez-Revuelta (CDC4CV’13) 16       Straightforward fusion of skeleton and silhouette Improvement in the recognition rate Include also side and top projected silhouettes Select the weight for each feature vector Feature subset selection Should we create a large bank of features and select them appropriately?
  • 17. 3rd IEEE Workshop on Consumer Depth Cameras for Computer Vision (CDC4CV) Sydney, December 2, 2013 ALEXANDROS A. CHAARAOUI JOSÉ R. PADILLA-LÓPEZ FRANCISCO FLÓREZ-REVUELTA

×