Introduction to
Action Recognition
in Python
@wideio
Bertrand NOUVEL, bn@wide.io
Jonathan KELSEY, jk@wide.io
Bernard HERNA...
WIDE IO - DEMOCRATISING THE BEST ALGORITHMS
INTRODUCTION TO ACTION RECOGNITION
PYDATA LONDON 2014 - BERNARD HERNANDEZ - JO...
WIDE IO - DEMOCRATISING THE BEST ALGORITHMS
INTRODUCTION TO ACTION RECOGNITION
PYDATA LONDON 2014 - BERNARD HERNANDEZ - JO...
WIDE IO - DEMOCRATISING THE BEST ALGORITHMS
INTRODUCTION TO ACTION RECOGNITION
PYDATA LONDON 2014 - BERNARD HERNANDEZ - JO...
WIDE IO - DEMOCRATISING THE BEST ALGORITHMS
INTRODUCTION TO ACTION RECOGNITION
PYDATA LONDON 2014 - BERNARD HERNANDEZ - JO...
WIDE IO - DEMOCRATISING THE BEST ALGORITHMS
INTRODUCTION TO ACTION RECOGNITION
PYDATA LONDON 2014 - BERNARD HERNANDEZ - JO...
WIDE IO - DEMOCRATISING THE BEST ALGORITHMS
INTRODUCTION TO ACTION RECOGNITION
PYDATA LONDON 2014 - BERNARD HERNANDEZ - JO...
WIDE IO - DEMOCRATISING THE BEST ALGORITHMS
INTRODUCTION TO ACTION RECOGNITION
PYDATA LONDON 2014 - BERNARD HERNANDEZ - JO...
WIDE IO - DEMOCRATISING THE BEST ALGORITHMS
INTRODUCTION TO ACTION RECOGNITION
PYDATA LONDON 2014 - BERNARD HERNANDEZ - JO...
WIDE IO - DEMOCRATISING THE BEST ALGORITHMS
INTRODUCTION TO ACTION RECOGNITION
PYDATA LONDON 2014 - BERNARD HERNANDEZ - JO...
WIDE IO - DEMOCRATISING THE BEST ALGORITHMS
INTRODUCTION TO ACTION RECOGNITION
PYDATA LONDON 2014 - BERNARD HERNANDEZ - JO...
WIDE IO - DEMOCRATISING THE BEST ALGORITHMS
INTRODUCTION TO ACTION RECOGNITION
PYDATA LONDON 2014 - BERNARD HERNANDEZ - JO...
PART I
PROCESSING
VIDEOS
TOWARDS FRAMEWORKS
WIDE IO - DEMOCRATISING THE BEST ALGORITHMS
INTRODUCTION TO ACTION RECOGNITION
PYDATA LONDON 2014 - BERNARD HERNANDEZ - JO...
WIDE IO - DEMOCRATISING THE BEST ALGORITHMS
INTRODUCTION TO ACTION RECOGNITION
PYDATA LONDON 2014 - BERNARD HERNANDEZ - JO...
WIDE IO - DEMOCRATISING THE BEST ALGORITHMS
INTRODUCTION TO ACTION RECOGNITION
PYDATA LONDON 2014 - BERNARD HERNANDEZ - JO...
WIDE IO - DEMOCRATISING THE BEST ALGORITHMS
INTRODUCTION TO ACTION RECOGNITION
PYDATA LONDON 2014 - BERNARD HERNANDEZ - JO...
WIDE IO - DEMOCRATISING THE BEST ALGORITHMS
INTRODUCTION TO ACTION RECOGNITION
PYDATA LONDON 2014 - BERNARD HERNANDEZ - JO...
WIDE IO - DEMOCRATISING THE BEST ALGORITHMS
INTRODUCTION TO ACTION RECOGNITION
PYDATA LONDON 2014 - BERNARD HERNANDEZ - JO...
PART II
UNDERSTANDING THE KEY
ELEMENTS OF THE PIPELINE
WIDE IO - DEMOCRATISING THE BEST ALGORITHMS
INTRODUCTION TO ACTION RECOGNITION
PYDATA LONDON 2014 - BERNARD HERNANDEZ - JO...
WIDE IO - DEMOCRATISING THE BEST ALGORITHMS
INTRODUCTION TO ACTION RECOGNITION
PYDATA LONDON 2014 - BERNARD HERNANDEZ - JO...
WIDE IO - DEMOCRATISING THE BEST ALGORITHMS
INTRODUCTION TO ACTION RECOGNITION
PYDATA LONDON 2014 - BERNARD HERNANDEZ - JO...
WIDE IO - DEMOCRATISING THE BEST ALGORITHMS
INTRODUCTION TO ACTION RECOGNITION
PYDATA LONDON 2014 - BERNARD HERNANDEZ - JO...
WIDE IO - DEMOCRATISING THE BEST ALGORITHMS
INTRODUCTION TO ACTION RECOGNITION
PYDATA LONDON 2014 - BERNARD HERNANDEZ - JO...
WIDE IO - DEMOCRATISING THE BEST ALGORITHMS
INTRODUCTION TO ACTION RECOGNITION
PYDATA LONDON 2014 - BERNARD HERNANDEZ - JO...
WIDE IO - DEMOCRATISING THE BEST ALGORITHMS
INTRODUCTION TO ACTION RECOGNITION
PYDATA LONDON 2014 - BERNARD HERNANDEZ - JO...
WIDE IO - DEMOCRATISING THE BEST ALGORITHMS
INTRODUCTION TO ACTION RECOGNITION
PYDATA LONDON 2014 - BERNARD HERNANDEZ - JO...
WIDE IO - DEMOCRATISING THE BEST ALGORITHMS
INTRODUCTION TO ACTION RECOGNITION
PYDATA LONDON 2014 - BERNARD HERNANDEZ - JO...
PART III
PUTTING EVERYTHING
TOGETHER
WIDE IO - DEMOCRATISING THE BEST ALGORITHMS
INTRODUCTION TO ACTION RECOGNITION
PYDATA LONDON 2014 - BERNARD HERNANDEZ - JO...
WIDE IO - DEMOCRATISING THE BEST ALGORITHMS
INTRODUCTION TO ACTION RECOGNITION
PYDATA LONDON 2014 - BERNARD HERNANDEZ - JO...
WIDE IO - DEMOCRATISING THE BEST ALGORITHMS
INTRODUCTION TO ACTION RECOGNITION
PYDATA LONDON 2014 - BERNARD HERNANDEZ - JO...
Introduction to
Action Recognition
in Python
@wideio
Bertrand NOUVEL, bn@wide.io
Jonathan KELSEY, jk@wide.io
Bernard HERNA...
Upcoming SlideShare
Loading in …5
×

Introduction to Action Recognition in Python by Bertrand Nouvel, Jonathan Kelsey, and Bernard Hernandez

1,501 views

Published on

Introduction to Action Recognition in Python by Bertrand Nouvel, Jonathan Kelsey, and Bernard Hernandez

Published in: Technology, Education
0 Comments
2 Likes
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total views
1,501
On SlideShare
0
From Embeds
0
Number of Embeds
26
Actions
Shares
0
Downloads
52
Comments
0
Likes
2
Embeds 0
No embeds

No notes for slide

Introduction to Action Recognition in Python by Bertrand Nouvel, Jonathan Kelsey, and Bernard Hernandez

  1. 1. Introduction to Action Recognition in Python @wideio Bertrand NOUVEL, bn@wide.io Jonathan KELSEY, jk@wide.io Bernard HERNANDEZ, bh@wide.io PYDATA LONDON 2014
  2. 2. WIDE IO - DEMOCRATISING THE BEST ALGORITHMS INTRODUCTION TO ACTION RECOGNITION PYDATA LONDON 2014 - BERNARD HERNANDEZ - JONATHAN KESLEY - BERTRAND NOUVEL Outline - (While you download the data) Forewords & Overview - PART 1: Video-processing in python - PART 2: The pipeline in details - PART 3: Putting it together
  3. 3. WIDE IO - DEMOCRATISING THE BEST ALGORITHMS INTRODUCTION TO ACTION RECOGNITION PYDATA LONDON 2014 - BERNARD HERNANDEZ - JONATHAN KESLEY - BERTRAND NOUVEL FOREWORDS
  4. 4. WIDE IO - DEMOCRATISING THE BEST ALGORITHMS INTRODUCTION TO ACTION RECOGNITION PYDATA LONDON 2014 - BERNARD HERNANDEZ - JONATHAN KESLEY - BERTRAND NOUVEL GETTING READY Python 2.x / 3.x Numy/Scipy PIL OpenCV2 ANACONDA WAKARI GET THE ABSOLUTELY ESSENTIAL PACKAGES : 1) 2) GET THE SOURCE CODE git clone https://bitbucket.org/wideio/pydata.git
  5. 5. WIDE IO - DEMOCRATISING THE BEST ALGORITHMS INTRODUCTION TO ACTION RECOGNITION PYDATA LONDON 2014 - BERNARD HERNANDEZ - JONATHAN KESLEY - BERTRAND NOUVEL WHO WE ARE WIDE IO - Democratising the best algorithms startup + consultancy
  6. 6. WIDE IO - DEMOCRATISING THE BEST ALGORITHMS INTRODUCTION TO ACTION RECOGNITION PYDATA LONDON 2014 - BERNARD HERNANDEZ - JONATHAN KESLEY - BERTRAND NOUVEL Why do Computer Vision in Python? Computer vision is difficult. Multiparadigm Best community Lots of packages Operator overloading Readable Reflexive Lightweight Python is 1000 slower than C++ “””
  7. 7. WIDE IO - DEMOCRATISING THE BEST ALGORITHMS INTRODUCTION TO ACTION RECOGNITION PYDATA LONDON 2014 - BERNARD HERNANDEZ - JONATHAN KESLEY - BERTRAND NOUVEL ACTION RECOGNITION
  8. 8. WIDE IO - DEMOCRATISING THE BEST ALGORITHMS INTRODUCTION TO ACTION RECOGNITION PYDATA LONDON 2014 - BERNARD HERNANDEZ - JONATHAN KESLEY - BERTRAND NOUVEL What is action recognition ? SEMANTIC GAP KTH, Human action dataset (Laptev) Classify actions Classification task
  9. 9. WIDE IO - DEMOCRATISING THE BEST ALGORITHMS INTRODUCTION TO ACTION RECOGNITION PYDATA LONDON 2014 - BERNARD HERNANDEZ - JONATHAN KESLEY - BERTRAND NOUVEL Types of Systems ACTION SPECIFIC HOLISTIC APPROACHES FEATURE BASED APPEARANCE BASED DEEP LEARNING Many priors Less priors
  10. 10. WIDE IO - DEMOCRATISING THE BEST ALGORITHMS INTRODUCTION TO ACTION RECOGNITION PYDATA LONDON 2014 - BERNARD HERNANDEZ - JONATHAN KESLEY - BERTRAND NOUVEL The Very-Traditional Pipeline RAW DATA FEATURE-EXTRACTION MACHINE LEARNING
  11. 11. WIDE IO - DEMOCRATISING THE BEST ALGORITHMS INTRODUCTION TO ACTION RECOGNITION PYDATA LONDON 2014 - BERNARD HERNANDEZ - JONATHAN KESLEY - BERTRAND NOUVEL The Very-Traditional Pipeline VIDEO FRAMES SPARSIFICATION CLASSIFICATION METHOD
  12. 12. WIDE IO - DEMOCRATISING THE BEST ALGORITHMS INTRODUCTION TO ACTION RECOGNITION PYDATA LONDON 2014 - BERNARD HERNANDEZ - JONATHAN KESLEY - BERTRAND NOUVEL The Very-Traditional Pipeline VIDEO FRAMES BAGS OF KEYPOINTS SVM
  13. 13. PART I PROCESSING VIDEOS TOWARDS FRAMEWORKS
  14. 14. WIDE IO - DEMOCRATISING THE BEST ALGORITHMS INTRODUCTION TO ACTION RECOGNITION PYDATA LONDON 2014 - BERNARD HERNANDEZ - JONATHAN KESLEY - BERTRAND NOUVEL - PYFFMPEG (outdated) - OPENCV - MLT - ON WAKARI Reading Videos anaconda$ vi io/player_mlt.py
  15. 15. WIDE IO - DEMOCRATISING THE BEST ALGORITHMS INTRODUCTION TO ACTION RECOGNITION PYDATA LONDON 2014 - BERNARD HERNANDEZ - JONATHAN KESLEY - BERTRAND NOUVEL - PYFFMPEG (outdated) - OPENCV - MLT - ON WAKARI Reading Videos anaconda$ vi io/player_cv.py
  16. 16. WIDE IO - DEMOCRATISING THE BEST ALGORITHMS INTRODUCTION TO ACTION RECOGNITION PYDATA LONDON 2014 - BERNARD HERNANDEZ - JONATHAN KESLEY - BERTRAND NOUVEL Displaying Images Recommended Alternatives anaconda$ python io/display_pyglet.py
  17. 17. WIDE IO - DEMOCRATISING THE BEST ALGORITHMS INTRODUCTION TO ACTION RECOGNITION PYDATA LONDON 2014 - BERNARD HERNANDEZ - JONATHAN KESLEY - BERTRAND NOUVEL Harris Corner Detector Detecting interest points: imsmooth=scipy.ndimage.gaussian_filter def harris(I,alpha=0.04,si=1): Ix,Iy = scipy.gradient(I) H11 = imsmooth(Ix*Ix, si) H12 = imsmooth(Ix*Iy, si) H22 = imsmooth(Iy*Iy, si) return ((H11*H22 - H12**2) - alpha*(H11+H22)**2) anaconda$ python keypoints/harris.py
  18. 18. WIDE IO - DEMOCRATISING THE BEST ALGORITHMS INTRODUCTION TO ACTION RECOGNITION PYDATA LONDON 2014 - BERNARD HERNANDEZ - JONATHAN KESLEY - BERTRAND NOUVEL Optical flow - wrapper based code anaconda$ python keyponats/custom_feature.py
  19. 19. WIDE IO - DEMOCRATISING THE BEST ALGORITHMS INTRODUCTION TO ACTION RECOGNITION PYDATA LONDON 2014 - BERNARD HERNANDEZ - JONATHAN KESLEY - BERTRAND NOUVEL Use the two previous elements to compute keypoints that contain information about the movement Ideas for extension: Make the same with a pyramidal approach. Spatio-temporal keypoints anaconda$ python keyponats/custom_feature.py
  20. 20. PART II UNDERSTANDING THE KEY ELEMENTS OF THE PIPELINE
  21. 21. WIDE IO - DEMOCRATISING THE BEST ALGORITHMS INTRODUCTION TO ACTION RECOGNITION PYDATA LONDON 2014 - BERNARD HERNANDEZ - JONATHAN KESLEY - BERTRAND NOUVEL Feature extractions
  22. 22. WIDE IO - DEMOCRATISING THE BEST ALGORITHMS INTRODUCTION TO ACTION RECOGNITION PYDATA LONDON 2014 - BERNARD HERNANDEZ - JONATHAN KESLEY - BERTRAND NOUVEL SIFT (128) SURF (64)Descriptor vector Feature extractions
  23. 23. WIDE IO - DEMOCRATISING THE BEST ALGORITHMS INTRODUCTION TO ACTION RECOGNITION PYDATA LONDON 2014 - BERNARD HERNANDEZ - JONATHAN KESLEY - BERTRAND NOUVEL Algorithms: Type (density, connectivity, …) Dimension (descriptor, position, …) Recursive clustering Connectivity (ward) Centroid (k-means) Density (optics)
  24. 24. WIDE IO - DEMOCRATISING THE BEST ALGORITHMS INTRODUCTION TO ACTION RECOGNITION PYDATA LONDON 2014 - BERNARD HERNANDEZ - JONATHAN KESLEY - BERTRAND NOUVEL
  25. 25. WIDE IO - DEMOCRATISING THE BEST ALGORITHMS INTRODUCTION TO ACTION RECOGNITION PYDATA LONDON 2014 - BERNARD HERNANDEZ - JONATHAN KESLEY - BERTRAND NOUVEL
  26. 26. WIDE IO - DEMOCRATISING THE BEST ALGORITHMS INTRODUCTION TO ACTION RECOGNITION PYDATA LONDON 2014 - BERNARD HERNANDEZ - JONATHAN KESLEY - BERTRAND NOUVEL
  27. 27. WIDE IO - DEMOCRATISING THE BEST ALGORITHMS INTRODUCTION TO ACTION RECOGNITION PYDATA LONDON 2014 - BERNARD HERNANDEZ - JONATHAN KESLEY - BERTRAND NOUVEL Support Vector Machines
  28. 28. WIDE IO - DEMOCRATISING THE BEST ALGORITHMS INTRODUCTION TO ACTION RECOGNITION PYDATA LONDON 2014 - BERNARD HERNANDEZ - JONATHAN KESLEY - BERTRAND NOUVEL Support Vector Machines Weighted-Support Vector Machines Different support
  29. 29. WIDE IO - DEMOCRATISING THE BEST ALGORITHMS INTRODUCTION TO ACTION RECOGNITION PYDATA LONDON 2014 - BERNARD HERNANDEZ - JONATHAN KESLEY - BERTRAND NOUVEL Support Vector Machines Weighted-Support Vector Machines Different support Different relevance (outliers)
  30. 30. PART III PUTTING EVERYTHING TOGETHER
  31. 31. WIDE IO - DEMOCRATISING THE BEST ALGORITHMS INTRODUCTION TO ACTION RECOGNITION PYDATA LONDON 2014 - BERNARD HERNANDEZ - JONATHAN KESLEY - BERTRAND NOUVEL Feature detection Clustering PCA Projection BOW Eigen Vectors & Mean Graphic Model Model.model centers
  32. 32. WIDE IO - DEMOCRATISING THE BEST ALGORITHMS INTRODUCTION TO ACTION RECOGNITION PYDATA LONDON 2014 - BERNARD HERNANDEZ - JONATHAN KESLEY - BERTRAND NOUVEL Distance of projection to BOW cluster centers with metric Mean of stack to create histogram Normalise (Area=1) Invert with 'discriminator' to turn into weighting PCA project with graphic model Feature detection stack for image
  33. 33. WIDE IO - DEMOCRATISING THE BEST ALGORITHMS INTRODUCTION TO ACTION RECOGNITION PYDATA LONDON 2014 - BERNARD HERNANDEZ - JONATHAN KESLEY - BERTRAND NOUVEL Training data Feature computer Feature stack Param grid Training Labels SVM predict all Testing labelsTesting data SVM auto train
  34. 34. Introduction to Action Recognition in Python @wideio Bertrand NOUVEL, bn@wide.io Jonathan KELSEY, jk@wide.io Bernard HERNANDEZ, bh@wide.io PYDATA LONDON 2014

×