3rd IEEE
Workshop
on
Consumer
Depth
Cameras
for
Computer
Vision
(CDC4CV)
Sydney,
December 2, 2013

ALEXANDROS A. CHAARAOUI...
1. Introduction
© A.A. Chaaraoui, J.R. Padilla-López and F. Flórez-Revuelta (CDC4CV’13)

2



Motivation:


Use of both ...
1. Introduction
© A.A. Chaaraoui, J.R. Padilla-López and F. Flórez-Revuelta (CDC4CV’13)

3



Motivation:


Use of both ...
1. Introduction
© A.A. Chaaraoui, J.R. Padilla-López and F. Flórez-Revuelta (CDC4CV’13)

4



Solution:


Fusing differe...
2. Fusion of skeleton and
silhouette
© A.A. Chaaraoui, J.R. Padilla-López and F. Flórez-Revuelta (CDC4CV’13)

5

Concatena...
3. Classification method based on
a bag of key poses
© A.A. Chaaraoui, J.R. Padilla-López and F. Flórez-Revuelta (CDC4CV’1...
3. Classification method based on
a bag of key poses
© A.A. Chaaraoui, J.R. Padilla-López and F. Flórez-Revuelta (CDC4CV’1...
3. Classification method based on
a bag of key poses
© A.A. Chaaraoui, J.R. Padilla-López and F. Flórez-Revuelta (CDC4CV’1...
© A.A. Chaaraoui, J.R. Padilla-López and F. Flórez-Revuelta (CDC4CV’13)

4. Experimentation

9



Evaluation with the MSR...
4. Experimentation
© A.A. Chaaraoui, J.R. Padilla-López and F. Flórez-Revuelta (CDC4CV’13)

10



Cross-subject validatio...
4. Experimentation
© A.A. Chaaraoui, J.R. Padilla-López and F. Flórez-Revuelta (CDC4CV’13)

11



Cross-subject validatio...
4. Experimentation
© A.A. Chaaraoui, J.R. Padilla-López and F. Flórez-Revuelta (CDC4CV’13)

12



Confusion matrices for ...
© A.A. Chaaraoui, J.R. Padilla-López and F. Flórez-Revuelta (CDC4CV’13)

4. Experimentation

13



Leave-one-actor-out:
5. Conclusions and future work
© A.A. Chaaraoui, J.R. Padilla-López and F. Flórez-Revuelta (CDC4CV’13)

14










...
5. Conclusions and future work
© A.A. Chaaraoui, J.R. Padilla-López and F. Flórez-Revuelta (CDC4CV’13)

15



We have alr...
5. Conclusions and future work
© A.A. Chaaraoui, J.R. Padilla-López and F. Flórez-Revuelta (CDC4CV’13)

16










...
3rd IEEE
Workshop
on
Consumer
Depth
Cameras
for
Computer
Vision
(CDC4CV)
Sydney,
December 2, 2013

ALEXANDROS A. CHAARAOUI...
Upcoming SlideShare
Loading in …5
×

Fusion of Skeletal and Silhouette-based Features for Human Action Recognition with RGB-D Devices

1,534 views
1,382 views

Published on

Paper presented at the 3rd Workshop on Consumer Depth Cameras for Computer Vision (CDC4CV), ICCV 2013 Workshop, Sydney (Australia), 2013

Published in: Technology, Business
0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total views
1,534
On SlideShare
0
From Embeds
0
Number of Embeds
1
Actions
Shares
0
Downloads
16
Comments
0
Likes
0
Embeds 0
No embeds

No notes for slide

Fusion of Skeletal and Silhouette-based Features for Human Action Recognition with RGB-D Devices

  1. 1. 3rd IEEE Workshop on Consumer Depth Cameras for Computer Vision (CDC4CV) Sydney, December 2, 2013 ALEXANDROS A. CHAARAOUI JOSÉ R. PADILLA-LÓPEZ FRANCISCO FLÓREZ-REVUELTA
  2. 2. 1. Introduction © A.A. Chaaraoui, J.R. Padilla-López and F. Flórez-Revuelta (CDC4CV’13) 2  Motivation:  Use of both skeleton and silhouette in previous works  Problems with skeleton: lack of precision or noisy caused by occlusion caused by body parts or objects Pick-up and Throw
  3. 3. 1. Introduction © A.A. Chaaraoui, J.R. Padilla-López and F. Flórez-Revuelta (CDC4CV’13) 3  Motivation:  Use of both skeleton and silhouette in previous works  Problems with silhouettes: the only available viewpoint is unfavourable for recognition Tennis Serve Forward Punch Hammer
  4. 4. 1. Introduction © A.A. Chaaraoui, J.R. Padilla-López and F. Flórez-Revuelta (CDC4CV’13) 4  Solution:  Fusing different features that complement each other: skeleton, RGB colour, silhouette (2D), volume (3D)…  In this work, we fuse skeleton and silhouette
  5. 5. 2. Fusion of skeleton and silhouette © A.A. Chaaraoui, J.R. Padilla-López and F. Flórez-Revuelta (CDC4CV’13) 5 Concatenation of skeleton and silhouette features  Skeleton:  Silhouette:  3D coordinates of the  joints 20 3  1 2 4 8 9 7 5 6 10 12 11 13 14 15 16 17 19 18 Radial summary
  6. 6. 3. Classification method based on a bag of key poses © A.A. Chaaraoui, J.R. Padilla-López and F. Flórez-Revuelta (CDC4CV’13) 6 [1] A.A. Chaaraoui, P. Climent-Pérez and F. Flórez-Revuelta. Silhouette-based human action recognition using sequences of key poses. Pattern Recognition Letters, 34(15):1799-1807,
  7. 7. 3. Classification method based on a bag of key poses © A.A. Chaaraoui, J.R. Padilla-López and F. Flórez-Revuelta (CDC4CV’13) 7 [1] A.A. Chaaraoui, P. Climent-Pérez and F. Flórez-Revuelta. Silhouette-based human action recognition using sequences of key poses. Pattern Recognition Letters, 34(15):1799-
  8. 8. 3. Classification method based on a bag of key poses © A.A. Chaaraoui, J.R. Padilla-López and F. Flórez-Revuelta (CDC4CV’13) 8  Sequence recognition Transform a sequence into a sequences of key poses using the bag of key poses  Sequence matching using dynamic time warping  [1] A.A. Chaaraoui, P. Climent-Pérez and F. Flórez-Revuelta. Silhouette-based human action recognition using sequences of key poses. Pattern Recognition Letters, 34(15):1799-1807,
  9. 9. © A.A. Chaaraoui, J.R. Padilla-López and F. Flórez-Revuelta (CDC4CV’13) 4. Experimentation 9  Evaluation with the MSR Action3D dataset
  10. 10. 4. Experimentation © A.A. Chaaraoui, J.R. Padilla-López and F. Flórez-Revuelta (CDC4CV’13) 10  Cross-subject validation as in [2]: Training: actors 1, 3, 5, 7 and 9  Testing: actors 2, 4, 6, 8 and 10  [2] W. Li, Z. Zhang, and Z. Liu. Action recognition based on a bag of 3D points. In 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition Workshops, pp. 9-14, 2010.
  11. 11. 4. Experimentation © A.A. Chaaraoui, J.R. Padilla-López and F. Flórez-Revuelta (CDC4CV’13) 11  Cross-subject validation as in [2]: [2] W. Li, Z. Zhang, and Z. Liu. Action recognition based on a bag of 3D points. In 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition Workshops, pp. 9-14, 2010.
  12. 12. 4. Experimentation © A.A. Chaaraoui, J.R. Padilla-López and F. Flórez-Revuelta (CDC4CV’13) 12  Confusion matrices for AS1: a02 a03 a05 a06 a10 a13 a18 a20 a02 0,92 0,08 a03 1,00 a05 0,91 0,09 a06 0,09 0,73 0,18 a10 1,00 a13 1,00 a18 1,00 a20 0,14 0,07 0,29 0,50 a02 a03 a05 a06 a10 a13 a18 a20 a02 0,67 0,25 0,08 a03 0,58 0,42 a05 0,18 0,73 0,09 a06 0,18 0,82 a10 1,00 a13 0,07 0,93 a18 0,33 0,20 0,07 0,40 a20 0,07 0,14 0,07 0,14 0,57 Skeleton Silhouette a02 a03 a05 a06 a10 a13 a18 a20 a02 1,00 a03 1,00 a05 0,09 0,91 a06 0,18 0,73 0,09 a10 1,00 a13 1,00 a18 1,00 a20 0,29 0,71 Fusion
  13. 13. © A.A. Chaaraoui, J.R. Padilla-López and F. Flórez-Revuelta (CDC4CV’13) 4. Experimentation 13  Leave-one-actor-out:
  14. 14. 5. Conclusions and future work © A.A. Chaaraoui, J.R. Padilla-López and F. Flórez-Revuelta (CDC4CV’13) 14      Straightforward fusion of skeleton and silhouette Improvement in the recognition rate Include also side and top projected silhouettes Select the weight for each feature vector Feature subset selection
  15. 15. 5. Conclusions and future work © A.A. Chaaraoui, J.R. Padilla-López and F. Flórez-Revuelta (CDC4CV’13) 15  We have already applied the approach in [3] for feature selection to the fusion of skeleton and silhouette Cross-Subject LOAO [3] A.A. Chaaraoui, J.R. Padilla-López, P. Climent-Pérez, and F. Flórez-Revuelta. Evolutionary joint selection to improve human action recognition with RGB-D devices. Expert Systems with Applications, 41(3):786-794,2014.
  16. 16. 5. Conclusions and future work © A.A. Chaaraoui, J.R. Padilla-López and F. Flórez-Revuelta (CDC4CV’13) 16       Straightforward fusion of skeleton and silhouette Improvement in the recognition rate Include also side and top projected silhouettes Select the weight for each feature vector Feature subset selection Should we create a large bank of features and select them appropriately?
  17. 17. 3rd IEEE Workshop on Consumer Depth Cameras for Computer Vision (CDC4CV) Sydney, December 2, 2013 ALEXANDROS A. CHAARAOUI JOSÉ R. PADILLA-LÓPEZ FRANCISCO FLÓREZ-REVUELTA

×