2. 1. Problem statement
2. Application architecture
3. Database structure
4. Classifier creation workflow
5. Case study and results
6. Conclusions and future improvements
Table of Contents
3. Problem statement
• Machine learning – highly experimental, difficult to use without
much domain knowledge
• General-purpose image classification workflow, allowing users to
build models and publish them for usage from any device.
• Main features:
• Data acquisition, data cleaning, parameter tuning, feature extraction,
classifier creation, basic statistics and publishing.
• Case study: classification of traffic signs
4. Application architecture
• Three components that can be decoupled from
each-other at any level.
1. A Python image classification library.
• API for all the major functionalities
• Sqlite database, OpenCV, NumPy and scikit-learn
• Only part that’s not replaceable, forming the base
of the stack.
5. 2. Flask server
• Exposes API for interaction with the “imclas” library in the application.
• Uses Flask microframework to provide a REST service
• Split into several modules in order to allow easy extension:
• Mod_collections
• Mod_models
• Mod_root
• Could be replaced with any other WSGI-compliant server.
Application architecture (2)
6. 3. AngularJS web application
• Single page application
• Communicates with Flask middleware
• Provides UI access to components of the library
• Can be replaced by any kind of app
that can access the web, like mobile apps.
Application architecture (3)
7. • Several entities were
needed to persist classifiers
• Images stored as paths
instead of blobs for access
speed.
• Classifiers also stored as
filesystem paths, serialized
in “.model” files.
Database structure
8. • Based on work done by Csurka et al., Xerox Research Center, 2004
1. Data acquisition using Bing Image Search API, possibility to do manual clean-up after
download.
2. Feature extraction.
• Computer Vision – OpenCV Python bindings used.
• Bag-of-features model from text analytics.
• SIFT algorithm used for keypoint and feature extraction.
Classifier creation workflow (1)
9. 3. Feature clustering using K-means classifier
• Can be tuned from the UI, by specifying number of clusters
• Benefits from multiple cores / processors, takes a lot of time.
• Each image is represented then as a normalized histogram with k bins.
4. Classifier creation
• SVM classifier type, used with chi-squared kernel, saved in the db
together with some statistics and serialized (cPickle Python library).
• Train-test data ratio and other parameters sent from the UI
Classifier creation workflow (2)
10. • Original study done on 5 categories: bikes, planes, faces, cars
and backgrounds, very accurate results (~98%).
• Our study done on 6 different types of traffic signs.
Case study and results
11. • Original study: 1000 clusters, 0.5 train-test split, chi-squared
kernel. (Csurka et al. 2004)
• Our study: 100 clusters and 0.8 train-test split. Chi-squared kernel
as in the original. Total signs: 909 images, 725 train, 184 test
• Accuracy of ~70%, confusion matrix in previous slide.
• Binary experiment also done, 92% accuracy (no left turn signs,
stop signs), 50 clusters.
Case study and results (2)
12. • Easy-to-use and extensible framework.
• Take position and color into account for
classifier.
• K-means takes a long time, cluster computing
could help.
• Mobile app that uses camera to access
published models.
Thank you!
Conclusions and
improvements
