-

1. Real-world application of
computer vision and machine
learning in modern web applications

2. Arkadiy Pilguk
@apilguk

3. pentatonica.co

5. Image Credit: Apple

8. Two sets of requirements
Models Library

9. Client side
Library Requirements

10. Client side Real time
Library Requirements

11. Client side Real time
Library Requirements
Graph Like Models

12. Client side
Easy to get started with
Real time
Library Requirements
Graph Like Models

13. Model Requirements
Simple as possible API
// create face detector
const faceDetector = new FaceDetector();
// create identity detector
const idDetector = new IdentityDetector();
// return position of founded face
const facePosition = faceDetector.detect(image);
// position of founded identity
const idPosition = idDetector.detect(image);
Environment Invariance

14. Gaussian Blur Image Gradient Canny NMS Hough Transform Enhance PeaksCanny Hysteresis
Hough Transform Pipeline
Layer 1 Layer 2 Layer 3 Layer 4 Layer 5 Layer 6
500 x 500 500 x 500 500 x 500 500 x 500 500 x 500 500 x 500

17. DECK.GL

18. Session
WebGL
Output Tensor
Operation API
JS WASM
Graph
GammaCV Architecture

19. Current Platform Support
Desktop
Mobile

20. Performance
Canny Edges
Gaussian Blur
Face Detection
50-55
40-45
55-60
35-40
45-50
30-35
35-40
35-40
30-35
Jsfeat (JS) OpenCV.js (WASM) OpenCV (CPU)FPS
MacBook Pro 2017, input size 500x400
55-60
55-60
55-60
GammaCV (WeGL)

21. Future Goals
Speed up with
WebGL 2
Enhance current
algorithms
Text localization
in real time
Optical character
recognition
Barcode support
Downlevel Support
(via WASM, JS backends)

22. Machine Learning
Algorithm
PredictionData
Why Machine Learning?

23. Computer Vision feat Machine Learning?
Canny edge detector

24. Computer Vision based
feature extraction
ML algorithm Class prediction
Statistical Learning

25. All lines All possible rectangles
Geometry features filter Statistical model filter

26. Then apply our statistical model to merge a
candidates
With using of the sliding window we trying to
find all possible candidates

27. Model Tuning

28. Accuracy Benchmarks
Train 60% Test 20% Validate 20%

30. User
Interface

32. Conclusion

33. Open Source

34. Open Source

35. gammacv.com

36. Thank You
Arkadiy Pilguk
apilguk@gmail.com