Uploaded on

Preliminary slides providing an overview of the Python Computer Vision Framework.

Preliminary slides providing an overview of the Python Computer Vision Framework.

More in: Technology
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Be the first to comment
No Downloads

Views

Total Views
7,767
On Slideshare
0
From Embeds
0
Number of Embeds
3

Actions

Shares
Downloads
24
Comments
0
Likes
4

Embeds 0

No embeds

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
    No notes for slide

Transcript

  • 1. PyCVF Overview of the Python Computer Vision Framework (preliminary version) Bertrand NOUVEL Japanese French Laboratory for Informatics bertrand.nouvel@gmail.com May 2010    
  • 2. Traditional Framework Machine Create Compute Extract Compute Fusion Learning Video DB Shots Keyframes Features Rendering Indexing Program 1 Program 2 Program 3 Program 4 Program 5 Program 6 Program 7 ● Too many steps ● Many access to files : low performances ● Difficult to check and to run ● Not easy to perform testings    
  • 3. Models We Need to Represent ● Learning all along the process; ● Many class­specific learners; ● Link with highlevel ontologies ● Ability to provide top­down and bottom­up  communications      Too complex fort traditional frameworks ! → Need for improvement    
  • 4. Can We Do Better ? ● Avoid usage of archaic file formats ● Use better software design ● Inspire from what is meant to be ”easy” Blender OpenCV     PureData
  • 5. PyCVF is a Framework OPENCV WEKA Dependencies Libraries MPT Orange Toolkits (features) Marsyas R Framework PyCVF (uniformize concepts) Applications Analyze Mosaicing Synthetize Indexer Recognizer Textures Loop generator (use and extend the framework)    
  • 6. Essential PyCVF Concepts DB Database Models Nodes Machine Datatypes Structures Experiment Learning   Models  
  • 7. PyCVF Datatypes ● Images → 3d ­ Numpy Array ● Video → Video Player Object (play/pause/seek) ● Audio → Audio Player Object (play/pause/seek) ● Vectors → 1d ­ Numpy Array  ● Vectorset → 2d – Numpy Array Datatypes can solve the following questions ? ● How to store the data ? ● How to display the data ?    
  • 8. PyCVF Databases Supported operations ● Get Datatype ● List Items  DB ● Query ● List Keys ● List Labels ● Query Label    
  • 9. Example PyCVF Image Databases Image Databases Online Databases Database : Mathematical functions Cellular Automatas... Pseudo-database : device Folder-based Databases    
  • 10. Example Meta databases ● Limit : Limit the number of elements ● Transformed : Apply a transformation  ● Exploded : Obtain sub­elements ● TrainDB : Create a training associated with a seed ● TestDB : Create a test set associated with a seed ● Randomized : Shuffle the elements of a database ● Agglomerated : Merge databases together ● As_once : Return databases as elements     
  • 11. Nodes ● Processing units ● Dynamic typing ● Associated Storage ● Interconnectable ● Labels ● May raise Exceptions (cf. bag_of_words)    
  • 12. Model=PyCVF Dataflow pycvf_features_view -m ”image.laplace()-image.keypoints.harris()” Laplace Harris Corner Detector    
  • 13. PyCVF Dataflow pycvf_features_view -m ”image.laplace()-image.keypoints.harris()+image.watershed()” Laplace Harris Corner Detector WaterShed    
  • 14. Essential Applications db_view installdb model_run evaluate_db features_view PyCVF create_index confusion_matrix distance_matrix nearest_neighbors run_experiment remote_job_server compute_features    
  • 15. Evaluate Database pycvf_dbevaluator .... ● PyCVF include a  generic tool to help  you to label your  database ● Works with any  datatypes and any  database.    
  • 16. Observe Features pycvf_model_features_view –db 'image.kanji()' --model 'image.keypoints.sift()' ● Easy debugging ● Better intuition ● Portable    
  • 17. Nearest Neighbors Query pycvf_neareastneighbors_qtapp –db ”image.kanji()” -m ”free('x.ravel()')” -n 5 -b25 -s kanjiidx    
  • 18. Nearest Neighbors Query pycvf_neareastneighbors_panda –db ”image.kanji()” -m ”free('x.ravel()')” -n 5 -b25 -s kanjiidx    
  • 19. Extract and Analyze Nearest Neighbor Graphs pycvf_neareastneighbors_qtapp –db ”image.kanji()” -m ”free('x.ravel()')” -n 5 -b25 -s kanjiidx    
  • 20. Distance Matrices pycvf_distance_matrix –db ”from_trackfile('HOG')”    
  • 21. Making Experiments ● Easy sampling of  parameter space ● Automatic  parallelization of  runs ● Specific graphics ● Specific feature ● Minimization Performances of PyCVF of index queries    
  • 22. NEW DJANGO + PyCVF = pycvf_remote_job_server & python manage.py runserver &    
  • 23. Browsing Databases ● On­the­fly  transformations ● Easy queries to already  computed index    
  • 24. Viewing Element 1 by 1 ● On­the­fly  transformations ● Easy queries to already  computed index    
  • 25. Revision Plan 0.2 0.3 ● Do call for volunteers ● Add Features ● Ensure global design ● Improve feature  ● Easier install  stability ● Increase core­robustness ● Provide exhaustive set  of basic features ● Regression tests  ● Start add new features    
  • 26. See you soon    
  • 27. Included Wrappers ● PyFFMPEG is able to  PyFFMPEG play in realtime 121  videos at the same  time.    
  • 28. Included Wrappers Implements  ● OpenCV-Cython exhaustively C API  ● Numpy compatibility  ● Accessible pointers ● May be deprecated due  to new C++ wrappers    
  • 29. Included Wrappers ● Fast nearest neighbor  PySASH search ● Patented + NOT GPL    
  • 30. Included Wrappers ● Fast nearest neighbor  PyLSH search with LSH ● Different  implementations ● Still Buggy ● Contributions are  welcome    
  • 31. Included Wrappers PySIFT ● Numpy compatible  version of PySIFT ●