The document is an introduction to action recognition using Python, presented at PyData London 2014 by Wide IO, detailing a multi-part guide to video processing and action classification. It covers the necessary tools, algorithms, and methodologies for implementing action recognition, including feature extraction, machine learning techniques, and pipeline execution. Key topics include the integration of libraries like OpenCV, SVM methods, and different approaches for feature detection and classification in computer vision.

1. Introduction to
Action Recognition
in Python
@wideio
Bertrand NOUVEL, bn@wide.io
Jonathan KELSEY, jk@wide.io
Bernard HERNANDEZ, bh@wide.io
PYDATA LONDON 2014

2. WIDE IO - DEMOCRATISING THE BEST ALGORITHMS
INTRODUCTION TO ACTION RECOGNITION
PYDATA LONDON 2014 - BERNARD HERNANDEZ - JONATHAN KESLEY - BERTRAND NOUVEL
Outline
- (While you download the data) Forewords & Overview
- PART 1: Video-processing in python
- PART 2: The pipeline in details
- PART 3: Putting it together

3. WIDE IO - DEMOCRATISING THE BEST ALGORITHMS
INTRODUCTION TO ACTION RECOGNITION
PYDATA LONDON 2014 - BERNARD HERNANDEZ - JONATHAN KESLEY - BERTRAND NOUVEL
FOREWORDS

4. WIDE IO - DEMOCRATISING THE BEST ALGORITHMS
INTRODUCTION TO ACTION RECOGNITION
PYDATA LONDON 2014 - BERNARD HERNANDEZ - JONATHAN KESLEY - BERTRAND NOUVEL
GETTING READY
Python 2.x / 3.x
Numy/Scipy
PIL
OpenCV2
ANACONDA
WAKARI
GET THE ABSOLUTELY ESSENTIAL PACKAGES :
1)
2)
GET THE SOURCE CODE
git clone https://bitbucket.org/wideio/pydata.git

5. WIDE IO - DEMOCRATISING THE BEST ALGORITHMS
INTRODUCTION TO ACTION RECOGNITION
PYDATA LONDON 2014 - BERNARD HERNANDEZ - JONATHAN KESLEY - BERTRAND NOUVEL
WHO WE ARE
WIDE IO - Democratising the best algorithms
startup + consultancy

6. WIDE IO - DEMOCRATISING THE BEST ALGORITHMS
INTRODUCTION TO ACTION RECOGNITION
PYDATA LONDON 2014 - BERNARD HERNANDEZ - JONATHAN KESLEY - BERTRAND NOUVEL
Why do Computer Vision in Python?
Computer vision is difficult.
Multiparadigm
Best community
Lots of packages
Operator
overloading
Readable
Reflexive
Lightweight
Python is 1000 slower than C++
“””

7. WIDE IO - DEMOCRATISING THE BEST ALGORITHMS
INTRODUCTION TO ACTION RECOGNITION
PYDATA LONDON 2014 - BERNARD HERNANDEZ - JONATHAN KESLEY - BERTRAND NOUVEL
ACTION RECOGNITION

8. WIDE IO - DEMOCRATISING THE BEST ALGORITHMS
INTRODUCTION TO ACTION RECOGNITION
PYDATA LONDON 2014 - BERNARD HERNANDEZ - JONATHAN KESLEY - BERTRAND NOUVEL
What is action recognition ?
SEMANTIC GAP
KTH, Human action dataset (Laptev)
Classify actions
Classification task

9. WIDE IO - DEMOCRATISING THE BEST ALGORITHMS
INTRODUCTION TO ACTION RECOGNITION
PYDATA LONDON 2014 - BERNARD HERNANDEZ - JONATHAN KESLEY - BERTRAND NOUVEL
Types of Systems
ACTION SPECIFIC
HOLISTIC
APPROACHES
FEATURE BASED
APPEARANCE
BASED
DEEP LEARNING
Many priors Less priors

10. WIDE IO - DEMOCRATISING THE BEST ALGORITHMS
INTRODUCTION TO ACTION RECOGNITION
PYDATA LONDON 2014 - BERNARD HERNANDEZ - JONATHAN KESLEY - BERTRAND NOUVEL
The Very-Traditional Pipeline
RAW DATA FEATURE-EXTRACTION MACHINE LEARNING

11. WIDE IO - DEMOCRATISING THE BEST ALGORITHMS
INTRODUCTION TO ACTION RECOGNITION
PYDATA LONDON 2014 - BERNARD HERNANDEZ - JONATHAN KESLEY - BERTRAND NOUVEL
The Very-Traditional Pipeline
VIDEO
FRAMES
SPARSIFICATION
CLASSIFICATION
METHOD

12. WIDE IO - DEMOCRATISING THE BEST ALGORITHMS
INTRODUCTION TO ACTION RECOGNITION
PYDATA LONDON 2014 - BERNARD HERNANDEZ - JONATHAN KESLEY - BERTRAND NOUVEL
The Very-Traditional Pipeline
VIDEO
FRAMES
BAGS OF KEYPOINTS SVM

13. PART I
PROCESSING
VIDEOS
TOWARDS FRAMEWORKS

14. WIDE IO - DEMOCRATISING THE BEST ALGORITHMS
INTRODUCTION TO ACTION RECOGNITION
PYDATA LONDON 2014 - BERNARD HERNANDEZ - JONATHAN KESLEY - BERTRAND NOUVEL
- PYFFMPEG (outdated)
- OPENCV
- MLT
- ON WAKARI
Reading Videos
anaconda$ vi io/player_mlt.py

15. WIDE IO - DEMOCRATISING THE BEST ALGORITHMS
INTRODUCTION TO ACTION RECOGNITION
PYDATA LONDON 2014 - BERNARD HERNANDEZ - JONATHAN KESLEY - BERTRAND NOUVEL
- PYFFMPEG (outdated)
- OPENCV
- MLT
- ON WAKARI
Reading Videos
anaconda$ vi io/player_cv.py

16. WIDE IO - DEMOCRATISING THE BEST ALGORITHMS
INTRODUCTION TO ACTION RECOGNITION
PYDATA LONDON 2014 - BERNARD HERNANDEZ - JONATHAN KESLEY - BERTRAND NOUVEL
Displaying Images
Recommended Alternatives
anaconda$ python io/display_pyglet.py

17. WIDE IO - DEMOCRATISING THE BEST ALGORITHMS
INTRODUCTION TO ACTION RECOGNITION
PYDATA LONDON 2014 - BERNARD HERNANDEZ - JONATHAN KESLEY - BERTRAND NOUVEL
Harris Corner Detector
Detecting interest points:
imsmooth=scipy.ndimage.gaussian_filter
def harris(I,alpha=0.04,si=1):
Ix,Iy = scipy.gradient(I)
H11 = imsmooth(Ix*Ix, si)
H12 = imsmooth(Ix*Iy, si)
H22 = imsmooth(Iy*Iy, si)
return ((H11*H22 - H12**2)
- alpha*(H11+H22)**2)
anaconda$ python keypoints/harris.py

18. WIDE IO - DEMOCRATISING THE BEST ALGORITHMS
INTRODUCTION TO ACTION RECOGNITION
PYDATA LONDON 2014 - BERNARD HERNANDEZ - JONATHAN KESLEY - BERTRAND NOUVEL
Optical flow - wrapper based code
anaconda$ python keyponats/custom_feature.py

19. WIDE IO - DEMOCRATISING THE BEST ALGORITHMS
INTRODUCTION TO ACTION RECOGNITION
PYDATA LONDON 2014 - BERNARD HERNANDEZ - JONATHAN KESLEY - BERTRAND NOUVEL
Use the two previous elements to compute keypoints that contain information about the movement
Ideas for extension: Make the same with a pyramidal approach.
Spatio-temporal keypoints
anaconda$ python keyponats/custom_feature.py

20. PART II
UNDERSTANDING THE KEY
ELEMENTS OF THE PIPELINE

21. WIDE IO - DEMOCRATISING THE BEST ALGORITHMS
INTRODUCTION TO ACTION RECOGNITION
PYDATA LONDON 2014 - BERNARD HERNANDEZ - JONATHAN KESLEY - BERTRAND NOUVEL
Feature extractions

22. WIDE IO - DEMOCRATISING THE BEST ALGORITHMS
INTRODUCTION TO ACTION RECOGNITION
PYDATA LONDON 2014 - BERNARD HERNANDEZ - JONATHAN KESLEY - BERTRAND NOUVEL
SIFT (128)
SURF (64)Descriptor vector
Feature extractions

23. WIDE IO - DEMOCRATISING THE BEST ALGORITHMS
INTRODUCTION TO ACTION RECOGNITION
PYDATA LONDON 2014 - BERNARD HERNANDEZ - JONATHAN KESLEY - BERTRAND NOUVEL
Algorithms:
Type (density, connectivity, …)
Dimension (descriptor, position, …)
Recursive clustering
Connectivity (ward) Centroid (k-means) Density (optics)

24. WIDE IO - DEMOCRATISING THE BEST ALGORITHMS
INTRODUCTION TO ACTION RECOGNITION
PYDATA LONDON 2014 - BERNARD HERNANDEZ - JONATHAN KESLEY - BERTRAND NOUVEL

25. WIDE IO - DEMOCRATISING THE BEST ALGORITHMS
INTRODUCTION TO ACTION RECOGNITION
PYDATA LONDON 2014 - BERNARD HERNANDEZ - JONATHAN KESLEY - BERTRAND NOUVEL

26. WIDE IO - DEMOCRATISING THE BEST ALGORITHMS
INTRODUCTION TO ACTION RECOGNITION
PYDATA LONDON 2014 - BERNARD HERNANDEZ - JONATHAN KESLEY - BERTRAND NOUVEL

27. WIDE IO - DEMOCRATISING THE BEST ALGORITHMS
INTRODUCTION TO ACTION RECOGNITION
PYDATA LONDON 2014 - BERNARD HERNANDEZ - JONATHAN KESLEY - BERTRAND NOUVEL
Support Vector Machines

28. WIDE IO - DEMOCRATISING THE BEST ALGORITHMS
INTRODUCTION TO ACTION RECOGNITION
PYDATA LONDON 2014 - BERNARD HERNANDEZ - JONATHAN KESLEY - BERTRAND NOUVEL
Support Vector Machines
Weighted-Support Vector Machines
Different support

29. WIDE IO - DEMOCRATISING THE BEST ALGORITHMS
INTRODUCTION TO ACTION RECOGNITION
PYDATA LONDON 2014 - BERNARD HERNANDEZ - JONATHAN KESLEY - BERTRAND NOUVEL
Support Vector Machines
Weighted-Support Vector Machines
Different support
Different relevance (outliers)

30. PART III
PUTTING EVERYTHING
TOGETHER

31. WIDE IO - DEMOCRATISING THE BEST ALGORITHMS
INTRODUCTION TO ACTION RECOGNITION
PYDATA LONDON 2014 - BERNARD HERNANDEZ - JONATHAN KESLEY - BERTRAND NOUVEL
Feature detection
Clustering
PCA Projection
BOW
Eigen Vectors & Mean
Graphic Model
Model.model
centers

32. WIDE IO - DEMOCRATISING THE BEST ALGORITHMS
INTRODUCTION TO ACTION RECOGNITION
PYDATA LONDON 2014 - BERNARD HERNANDEZ - JONATHAN KESLEY - BERTRAND NOUVEL
Distance of projection to BOW cluster centers with metric
Mean of stack to create histogram
Normalise (Area=1)
Invert with 'discriminator' to turn into weighting
PCA project with graphic model
Feature detection stack for image

33. WIDE IO - DEMOCRATISING THE BEST ALGORITHMS
INTRODUCTION TO ACTION RECOGNITION
PYDATA LONDON 2014 - BERNARD HERNANDEZ - JONATHAN KESLEY - BERTRAND NOUVEL
Training data
Feature computer
Feature stack
Param grid
Training Labels
SVM predict all
Testing labelsTesting data
SVM auto train

34. Introduction to
Action Recognition
in Python
@wideio
Bertrand NOUVEL, bn@wide.io
Jonathan KELSEY, jk@wide.io
Bernard HERNANDEZ, bh@wide.io
PYDATA LONDON 2014