Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Wide IO Presentation PyData London

Tutorial from PyData London: An Introduction to Video Action Recognition by Bertrand Nouvel, Jonathan Kelsey and Bernard Hernandez from Wide IO

  • Be the first to comment

  • Be the first to like this

Wide IO Presentation PyData London

  1. 1. Introduction to Action Recognition in Python @wideio Bertrand NOUVEL, bn@wide.io Jonathan KELSEY, jk@wide.io Bernard HERNANDEZ, bh@wide.io PYDATA LONDON 2014
  2. 2. Outline - (While you download the data) Forewords & Overview - PART 1: Video-processing in python - PART 2: The pipeline in details - PART 3: Putting it together WIDE IO - DEMOCRATISING THE BEST ALGORITHMS INTRODUCTION TO ACTION RECOGNITION PYDATA LONDON 2014 - BERNARD HERNANDEZ - JONATHAN KESLEY - BERTRAND NOUVEL
  3. 3. FOREWORDS WIDE IO - DEMOCRATISING THE BEST ALGORITHMS INTRODUCTION TO ACTION RECOGNITION PYDATA LONDON 2014 - BERNARD HERNANDEZ - JONATHAN KESLEY - BERTRAND NOUVEL
  4. 4. GETTING READY 1) 2) GET THE ABSOLUTELY ESSENTIAL PACKAGES : Python 2.x / 3.x Numy/Scipy PIL OpenCV2 ANACONDA WAKARI GET THE SOURCE CODE git clone https://bitbucket.org/wideio/pydata.git WIDE IO - DEMOCRATISING THE BEST ALGORITHMS INTRODUCTION TO ACTION RECOGNITION PYDATA LONDON 2014 - BERNARD HERNANDEZ - JONATHAN KESLEY - BERTRAND NOUVEL
  5. 5. WHO WE ARE WIDE IO - Democratising the best algorithms startup + consultancy WIDE IO - DEMOCRATISING THE BEST ALGORITHMS INTRODUCTION TO ACTION RECOGNITION PYDATA LONDON 2014 - BERNARD HERNANDEZ - JONATHAN KESLEY - BERTRAND NOUVEL
  6. 6. Why do Computer Vision in Python? Reflexive Multiparadigm Computer vision is difficult. “”” Readable Python is 1000 slower than C++ Lots of packages Lightweight Best community Operator overloading WIDE IO - DEMOCRATISING THE BEST ALGORITHMS INTRODUCTION TO ACTION RECOGNITION PYDATA LONDON 2014 - BERNARD HERNANDEZ - JONATHAN KESLEY - BERTRAND NOUVEL
  7. 7. ACTION RECOGNITION WIDE IO - DEMOCRATISING THE BEST ALGORITHMS INTRODUCTION TO ACTION RECOGNITION PYDATA LONDON 2014 - BERNARD HERNANDEZ - JONATHAN KESLEY - BERTRAND NOUVEL
  8. 8. What is action recognition ? Classification task SEMANTIC GAP KTH, Human action dataset (Laptev) Classify actions WIDE IO - DEMOCRATISING THE BEST ALGORITHMS INTRODUCTION TO ACTION RECOGNITION PYDATA LONDON 2014 - BERNARD HERNANDEZ - JONATHAN KESLEY - BERTRAND NOUVEL
  9. 9. Types of Systems FEATURE BASED ACTION SPECIFIC HOLISTIC APPROACHES APPEARANCE BASED Many priors WIDE IO - DEMOCRATISING THE BEST ALGORITHMS INTRODUCTION TO ACTION RECOGNITION PYDATA LONDON 2014 - BERNARD HERNANDEZ - JONATHAN KESLEY - BERTRAND NOUVEL DEEP LEARNING Less priors
  10. 10. The Very-Traditional Pipeline RAW DATA FEATURE-EXTRACTION MACHINE LEARNING WIDE IO - DEMOCRATISING THE BEST ALGORITHMS INTRODUCTION TO ACTION RECOGNITION PYDATA LONDON 2014 - BERNARD HERNANDEZ - JONATHAN KESLEY - BERTRAND NOUVEL
  11. 11. The Very-Traditional Pipeline VIDEO FRAMES SPARSIFICATION CLASSIFICATION METHOD WIDE IO - DEMOCRATISING THE BEST ALGORITHMS INTRODUCTION TO ACTION RECOGNITION PYDATA LONDON 2014 - BERNARD HERNANDEZ - JONATHAN KESLEY - BERTRAND NOUVEL
  12. 12. The Very-Traditional Pipeline VIDEO FRAMES BAGS OF KEYPOINTS SVM WIDE IO - DEMOCRATISING THE BEST ALGORITHMS INTRODUCTION TO ACTION RECOGNITION PYDATA LONDON 2014 - BERNARD HERNANDEZ - JONATHAN KESLEY - BERTRAND NOUVEL
  13. 13. PART I PROCESSING VIDEOS TOWARDS FRAMEWORKS
  14. 14. Reading Videos - PYFFMPEG (outdated) - OPENCV MLT - ON WAKARI anaconda$ vi io/player_mlt.py WIDE IO - DEMOCRATISING THE BEST ALGORITHMS INTRODUCTION TO ACTION RECOGNITION PYDATA LONDON 2014 - BERNARD HERNANDEZ - JONATHAN KESLEY - BERTRAND NOUVEL
  15. 15. Reading Videos - PYFFMPEG (outdated) - OPENCV MLT - ON WAKARI anaconda$ vi io/player_cv.py WIDE IO - DEMOCRATISING THE BEST ALGORITHMS INTRODUCTION TO ACTION RECOGNITION PYDATA LONDON 2014 - BERNARD HERNANDEZ - JONATHAN KESLEY - BERTRAND NOUVEL
  16. 16. Displaying Images Alternatives Recommended anaconda$ python io/display_pyglet.py WIDE IO - DEMOCRATISING THE BEST ALGORITHMS INTRODUCTION TO ACTION RECOGNITION PYDATA LONDON 2014 - BERNARD HERNANDEZ - JONATHAN KESLEY - BERTRAND NOUVEL
  17. 17. Harris Corner Detector Detecting interest points: imsmooth=scipy.ndimage.gaussian_filter def harris(I,alpha=0.04,si=1): Ix,Iy = scipy.gradient(I) H11 = imsmooth(Ix*Ix, si) H12 = imsmooth(Ix*Iy, si) H22 = imsmooth(Iy*Iy, si) return ((H11*H22 - H12**2) - alpha*(H11+H22)**2) anaconda$ python keypoints/harris.py WIDE IO - DEMOCRATISING THE BEST ALGORITHMS INTRODUCTION TO ACTION RECOGNITION PYDATA LONDON 2014 - BERNARD HERNANDEZ - JONATHAN KESLEY - BERTRAND NOUVEL
  18. 18. Optical flow - wrapper based code anaconda$ python keyponats/custom_feature.py WIDE IO - DEMOCRATISING THE BEST ALGORITHMS INTRODUCTION TO ACTION RECOGNITION PYDATA LONDON 2014 - BERNARD HERNANDEZ - JONATHAN KESLEY - BERTRAND NOUVEL
  19. 19. Spatio-temporal keypoints Use the two previous elements to compute keypoints that contain information about the movement anaconda$ python keyponats/custom_feature.py Ideas for extension: Make the same with a pyramidal approach. WIDE IO - DEMOCRATISING THE BEST ALGORITHMS INTRODUCTION TO ACTION RECOGNITION PYDATA LONDON 2014 - BERNARD HERNANDEZ - JONATHAN KESLEY - BERTRAND NOUVEL
  20. 20. PART II UNDERSTANDING THE KEY ELEMENTS OF THE PIPELINE
  21. 21. Feature extractions WIDE IO - DEMOCRATISING THE BEST ALGORITHMS INTRODUCTION TO ACTION RECOGNITION PYDATA LONDON 2014 - BERNARD HERNANDEZ - JONATHAN KESLEY - BERTRAND NOUVEL
  22. 22. Feature extractions SIFT (128) Descriptor vector WIDE IO - DEMOCRATISING THE BEST ALGORITHMS INTRODUCTION TO ACTION RECOGNITION PYDATA LONDON 2014 - BERNARD HERNANDEZ - JONATHAN KESLEY - BERTRAND NOUVEL SURF (64)
  23. 23. Algorithms: Type (density, connectivity, …) Dimension (descriptor, position, …) Recursive clustering Connectivity (ward) Centroid (k-means) WIDE IO - DEMOCRATISING THE BEST ALGORITHMS INTRODUCTION TO ACTION RECOGNITION PYDATA LONDON 2014 - BERNARD HERNANDEZ - JONATHAN KESLEY - BERTRAND NOUVEL Density (optics)
  24. 24. WIDE IO - DEMOCRATISING THE BEST ALGORITHMS INTRODUCTION TO ACTION RECOGNITION PYDATA LONDON 2014 - BERNARD HERNANDEZ - JONATHAN KESLEY - BERTRAND NOUVEL
  25. 25. WIDE IO - DEMOCRATISING THE BEST ALGORITHMS INTRODUCTION TO ACTION RECOGNITION PYDATA LONDON 2014 - BERNARD HERNANDEZ - JONATHAN KESLEY - BERTRAND NOUVEL
  26. 26. WIDE IO - DEMOCRATISING THE BEST ALGORITHMS INTRODUCTION TO ACTION RECOGNITION PYDATA LONDON 2014 - BERNARD HERNANDEZ - JONATHAN KESLEY - BERTRAND NOUVEL
  27. 27. Support Vector Machines WIDE IO - DEMOCRATISING THE BEST ALGORITHMS INTRODUCTION TO ACTION RECOGNITION PYDATA LONDON 2014 - BERNARD HERNANDEZ - JONATHAN KESLEY - BERTRAND NOUVEL
  28. 28. Support Vector Machines Weighted-Support Vector Machines Different support WIDE IO - DEMOCRATISING THE BEST ALGORITHMS INTRODUCTION TO ACTION RECOGNITION PYDATA LONDON 2014 - BERNARD HERNANDEZ - JONATHAN KESLEY - BERTRAND NOUVEL
  29. 29. Support Vector Machines Weighted-Support Vector Machines Different support Different relevance (outliers) WIDE IO - DEMOCRATISING THE BEST ALGORITHMS INTRODUCTION TO ACTION RECOGNITION PYDATA LONDON 2014 - BERNARD HERNANDEZ - JONATHAN KESLEY - BERTRAND NOUVEL
  30. 30. PART III PUTTING EVERYTHING TOGETHER
  31. 31. Feature detection PCA Projection Clustering Eigen Vectors & Mean Graphic Model Model.model centers BOW WIDE IO - DEMOCRATISING THE BEST ALGORITHMS INTRODUCTION TO ACTION RECOGNITION PYDATA LONDON 2014 - BERNARD HERNANDEZ - JONATHAN KESLEY - BERTRAND NOUVEL
  32. 32. Feature detection stack for image PCA project with graphic model Distance of projection to BOW cluster centers with metric Invert with 'discriminator' to turn into weighting Normalise (Area=1) Mean of stack to create histogram WIDE IO - DEMOCRATISING THE BEST ALGORITHMS INTRODUCTION TO ACTION RECOGNITION PYDATA LONDON 2014 - BERNARD HERNANDEZ - JONATHAN KESLEY - BERTRAND NOUVEL
  33. 33. Training data Training Labels Testing data Testing labels Feature computer SVM predict all Feature stack SVM auto train Param grid WIDE IO - DEMOCRATISING THE BEST ALGORITHMS INTRODUCTION TO ACTION RECOGNITION PYDATA LONDON 2014 - BERNARD HERNANDEZ - JONATHAN KESLEY - BERTRAND NOUVEL
  34. 34. Introduction to Action Recognition in Python @wideio Bertrand NOUVEL, bn@wide.io Jonathan KELSEY, jk@wide.io Bernard HERNANDEZ, bh@wide.io PYDATA LONDON 2014

×