CV勉強会 - CVPR2016読み会の資料

1. Convolutional Pose Machines
@conta_

2. 緒方 貴紀 (twitter: @conta_)
CTO@ABEJA, Inc.
Computer Visionとか、Machine Learningを使った
プロダクト開発をやっています。
Self Introduction

3. Pose Estimation?

4. Related Work
Pictorial structures Hierarchical models
Sequential prediction Convolutional architectures
[A. Toshev and C. Szegedy, CVPR’2013]
[Tian et al., ICCV’2011][Mykhaylo et al., CVPR’2009]
[Ramakrishna et al., 2014]

5. Related Work
Pictorial structures Hierarchical models
Sequential prediction Convolutional architectures
[A. Toshev and C. Szegedy, CVPR’2013]
[Tian et al., ICCV’2011][Mykhaylo et al., CVPR’2009]
[Ramakrishna et al., 2014]

6. Pose Machines?
Confidence Maps
[Ramakrishna et al., 2014]

7. Pose Machines
[Ramakrishna et al., 2014]
パッチから特徴量を抽出し、各Parts or NotのClassifier
を用いて、Confidence Mapsを作りたい

8. Pose Machines
[Ramakrishna et al., 2014]
局所的な特徴を用いた推定はWeak
Partsによっては、局所的特徴だど推定できない。。。

9. Part Contextは非常に有効な特徴
Pose Machines
[Ramakrishna et al., 2014]
前段階での推定結果を用いて、各Partsの関係性を
事前情報無しにを活用出来ないか？

10. Pose Machines
Stage I Confidence
Head Neck L-Shoulder L-Elbow L-Wrist
g2g1 g3
Context
Features
Context
Features
Stage I
Confidence Maps
Stage II
Confidence Maps
Stage III
Confidence Maps
Image
Features
[Ramakrishna et al., 2014]

11. Pose Machines
Stage II Confidence
g2g1 g3
Context
Features
Context
Features
Stage I
Confidence Maps
Stage II
Confidence Maps
Stage III
Confidence Maps
Image
Features
Head Neck L-Shoulder L-Elbow L-Wrist
[Ramakrishna et al., 2014]

12. Pose Machines
Stage III Confidence
Head Neck L-Shoulder L-Elbow L-Wrist
g2g1 g3
Context
Features
Context
Features
Stage I
Confidence Maps
Stage II
Confidence Maps
Stage III
Confidence Maps
Image
Features
Head Neck L-Shoulder L-Elbow L-Wrist
[Ramakrishna et al., 2014]

13. Pose Machines
Stage III Confidence
Head Neck L-Shoulder L-Elbow L-Wrist
g2g1 g3
Context
Features
Context
Features
Stage I
Confidence Maps
Stage II
Confidence Maps
Stage III
Confidence Maps
Image
Features
Head Neck L-Shoulder L-Elbow L-Wrist
[Ramakrishna et al., 2014]

14. Pose Machines (Previous Work)

15. Pose Machines (Previous Work)
HoG Random Forests

16. Convolutional Pose Machines (CPM)
Deep Deep

17. Architecture of CPM
階層的なCNNによるPose Machinesの実現

18. Stage1
Stage1: Localな特徴量の学習
368x368のInputに対して160x160の範囲をカバー

19. 各Parts + BackgroundのConfidence Maps (P+1)
＊MPII Human Pose Datasetだと P=14
Outputs

20. Stage 2
Stage2: Localな特徴量 + Part Contextによる学習

21. Stage T
Stage2と同じ構成のネットワークを積み上げていく
＊本研究ではStage6まで積み上げる

22. 各StageのConfidence Mapsと教師データとのEuclidean Distance Loss
教師データ: 各PartsのGround truth locationからGaussian Peakを計算したもの
Loss Function

23. Stage2以降、全段階のConfidence Mapのおかげで
良い推定ができている
Spatial context from belief maps

24. 3つのDatasetsで実験
- MPII Human Pose Dataset
- Leeds Sports Pose (LSP) Datase
- FLIC Dataset
Experiments

25. Results

26. Results
Stageは積み上げるとイイんやで

27. Results
Pose Machinesを上回る精度
Stageを重ねるごとに精度は向上

28. Results
State-of-the-art Performance (ﾄﾞﾔｧ

29. 実装してみた

30. 1. (色々頑張って実験した結果)いい感じの連続構成
CNNによって、暗黙的な空間モデルの学習ができた
2. Graphical Modelによる推論無しに、階層構造の
Predictionができた
Conclusion

31. We are hiring!
→ https://www.wantedly.com/
companies/abeja
博士持ち大歓迎！