Pycvf

8,300 views

Published on

Preliminary slides providing an overview of the Python Computer Vision Framework.

Published in: Technology
0 Comments
4 Likes
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total views
8,300
On SlideShare
0
From Embeds
0
Number of Embeds
6,495
Actions
Shares
0
Downloads
27
Comments
0
Likes
4
Embeds 0
No embeds

No notes for slide

Pycvf

  1. 1. PyCVF Overview of the Python Computer Vision Framework (preliminary version) Bertrand NOUVEL Japanese French Laboratory for Informatics bertrand.nouvel@gmail.com May 2010    
  2. 2. Traditional Framework Machine Create Compute Extract Compute Fusion Learning Video DB Shots Keyframes Features Rendering Indexing Program 1 Program 2 Program 3 Program 4 Program 5 Program 6 Program 7 ● Too many steps ● Many access to files : low performances ● Difficult to check and to run ● Not easy to perform testings    
  3. 3. Models We Need to Represent ● Learning all along the process; ● Many class­specific learners; ● Link with highlevel ontologies ● Ability to provide top­down and bottom­up  communications      Too complex fort traditional frameworks ! → Need for improvement    
  4. 4. Can We Do Better ? ● Avoid usage of archaic file formats ● Use better software design ● Inspire from what is meant to be ”easy” Blender OpenCV     PureData
  5. 5. PyCVF is a Framework OPENCV WEKA Dependencies Libraries MPT Orange Toolkits (features) Marsyas R Framework PyCVF (uniformize concepts) Applications Analyze Mosaicing Synthetize Indexer Recognizer Textures Loop generator (use and extend the framework)    
  6. 6. Essential PyCVF Concepts DB Database Models Nodes Machine Datatypes Structures Experiment Learning   Models  
  7. 7. PyCVF Datatypes ● Images → 3d ­ Numpy Array ● Video → Video Player Object (play/pause/seek) ● Audio → Audio Player Object (play/pause/seek) ● Vectors → 1d ­ Numpy Array  ● Vectorset → 2d – Numpy Array Datatypes can solve the following questions ? ● How to store the data ? ● How to display the data ?    
  8. 8. PyCVF Databases Supported operations ● Get Datatype ● List Items  DB ● Query ● List Keys ● List Labels ● Query Label    
  9. 9. Example PyCVF Image Databases Image Databases Online Databases Database : Mathematical functions Cellular Automatas... Pseudo-database : device Folder-based Databases    
  10. 10. Example Meta databases ● Limit : Limit the number of elements ● Transformed : Apply a transformation  ● Exploded : Obtain sub­elements ● TrainDB : Create a training associated with a seed ● TestDB : Create a test set associated with a seed ● Randomized : Shuffle the elements of a database ● Agglomerated : Merge databases together ● As_once : Return databases as elements     
  11. 11. Nodes ● Processing units ● Dynamic typing ● Associated Storage ● Interconnectable ● Labels ● May raise Exceptions (cf. bag_of_words)    
  12. 12. Model=PyCVF Dataflow pycvf_features_view -m ”image.laplace()-image.keypoints.harris()” Laplace Harris Corner Detector    
  13. 13. PyCVF Dataflow pycvf_features_view -m ”image.laplace()-image.keypoints.harris()+image.watershed()” Laplace Harris Corner Detector WaterShed    
  14. 14. Essential Applications db_view installdb model_run evaluate_db features_view PyCVF create_index confusion_matrix distance_matrix nearest_neighbors run_experiment remote_job_server compute_features    
  15. 15. Evaluate Database pycvf_dbevaluator .... ● PyCVF include a  generic tool to help  you to label your  database ● Works with any  datatypes and any  database.    
  16. 16. Observe Features pycvf_model_features_view –db 'image.kanji()' --model 'image.keypoints.sift()' ● Easy debugging ● Better intuition ● Portable    
  17. 17. Nearest Neighbors Query pycvf_neareastneighbors_qtapp –db ”image.kanji()” -m ”free('x.ravel()')” -n 5 -b25 -s kanjiidx    
  18. 18. Nearest Neighbors Query pycvf_neareastneighbors_panda –db ”image.kanji()” -m ”free('x.ravel()')” -n 5 -b25 -s kanjiidx    
  19. 19. Extract and Analyze Nearest Neighbor Graphs pycvf_neareastneighbors_qtapp –db ”image.kanji()” -m ”free('x.ravel()')” -n 5 -b25 -s kanjiidx    
  20. 20. Distance Matrices pycvf_distance_matrix –db ”from_trackfile('HOG')”    
  21. 21. Making Experiments ● Easy sampling of  parameter space ● Automatic  parallelization of  runs ● Specific graphics ● Specific feature ● Minimization Performances of PyCVF of index queries    
  22. 22. NEW DJANGO + PyCVF = pycvf_remote_job_server & python manage.py runserver &    
  23. 23. Browsing Databases ● On­the­fly  transformations ● Easy queries to already  computed index    
  24. 24. Viewing Element 1 by 1 ● On­the­fly  transformations ● Easy queries to already  computed index    
  25. 25. Revision Plan 0.2 0.3 ● Do call for volunteers ● Add Features ● Ensure global design ● Improve feature  ● Easier install  stability ● Increase core­robustness ● Provide exhaustive set  of basic features ● Regression tests  ● Start add new features    
  26. 26. See you soon    
  27. 27. Included Wrappers ● PyFFMPEG is able to  PyFFMPEG play in realtime 121  videos at the same  time.    
  28. 28. Included Wrappers Implements  ● OpenCV-Cython exhaustively C API  ● Numpy compatibility  ● Accessible pointers ● May be deprecated due  to new C++ wrappers    
  29. 29. Included Wrappers ● Fast nearest neighbor  PySASH search ● Patented + NOT GPL    
  30. 30. Included Wrappers ● Fast nearest neighbor  PyLSH search with LSH ● Different  implementations ● Still Buggy ● Contributions are  welcome    
  31. 31. Included Wrappers PySIFT ● Numpy compatible  version of PySIFT ●    

×