Mateusz Richtscheid: Wraz z rozwojem machine learningu zwiększają się zdolności maszyn do postrzegania rzeczywistości. Szczególnie popularna w dziedzinie wizji komputerowej jest klasyfikacja obiektów na zdjęciach. W trakcie prezentacji pokażę, na przykładzie serwera OpenCPU i sieci Inception, w jaki sposób wzbogacić projekt webowy o tę możliwość.

1. We must go deeper
Mariusz Richtscheid
Uszanowanko Programowanko
Gliwice, 31.08.2017
Image source: http://whatnext.pl/

2. About me
• Web Developer in The Software House
• richtscheid@gmail.com
• linkedin.com/in/richtscheid
• github.com/barograf
Mariusz Richtscheid
2

3. Presentation plan
• Some theory about neural networks
• Application demo
• How the app works
3

4. Motivation
Explicit programming Neural networks
Image source: www.newscientist.com, http://www.extremetech.com/4

5. Neuron
Neuron y
x1
x2
xm
Output value
Input values
Weights
w1
w2
wm
y = φ( iw ixi=1
m
∑ )
5
…
φ - activation function

6. Activation function -
sigmoid
iw ixi=1
m
∑
φ(x) =
1
1+
−x
e
y
0
1
6

7. Example - AND function (1/3)
1x , 2x ∈{0,1} y = 1x AND 2x
x1 x2 y
0 0
0 1
1 0
1 1
0
0
0
1
7

8. y
1
x1
x2
{0,1}
{0,1}
w0=?
w1=?
w2=?
y = φ( 0w + 1x 1w + 2x 2w )
8
y
Bias
Example - AND function (2/3)

9. y
1
x1
x2
w0=-30
w1=20
w2=20
y = φ(−30 + 20 1x + 20 2x )
x1 x2 y
0 0
0 1
1 0
1 1
φ(−30) ≈ 0
φ(−10) ≈ 0
φ(10) ≈1
φ(−10) ≈ 0
9
Example - AND function (3/3)

10. Neural network
y1
x1
x2
xm
Output layerInput layer
Hidden layer
y2
Weights
Weights
10
…

11. Image source: funnyjunk.com

12. But what does it mean?
• More neurons and layers
• Sophisticated models and learning algorithms
• Very long training times on high-end hardware
• Gigabytes of training data
• More network abilities
12

13. Convolutional neural
network
Image
Convolutional
neural network
Labels
13

14. Convolutional neural
network
Image source: http://www.wildml.com/14

15. Convolution layer (1/3)
Image source: http://setosa.io/ev/image-kernels/15

16. Convolution layer (2/3)
0 0 1 1 0 0
0 1 0 0 1 0
0 1 0 0 1 0
0 1 1 1 1 0
0 1 0 0 1 0
0 1 0 0 1 0
1 0 0
1 0 0
1 1 1
1 2 2 2
2 5 3 2
1 3 2 2
1 3 2 2
=*
Input image
Feature detector
Filter
Kernel
Feature map
Convolution layer
16

17. Convolution layer (3/3)
Image source: http://deeplearning.stanford.edu/17

18. Max pooling (1/3)
18
Feature map Pooled feature map

19. Max pooling (2/3)
1 2 2 2
2 5 3 2
1 3 2 2
1 3 2 2
Feature map Pooled feature map
5 3
3 2
19

20. Max pooling (3/3)
Image source: http://ufldl.stanford.edu/wiki/index.php/Pooling20

21. Full connection
5
3
3
Flattened feature map Neural network Labels
Dog
Hot-dog
21

22. Image source: http://scs.ryerson.ca/~aharley/vis/conv/flat.html

23. Demo
23

24. Project
• https://github.com/barograf/InceptionBot
• Customisable chat-bot called Hubot
• OpenCPU with MXNet library
• Inception network for object recognition
24

25. Architecture
Hubot
Camera
OpenCPU
with Inception
Slack
Docker icon source: docker.com25

26. Data flow
Camera OpenCPU
Look
Took a photo
I guess there is…
coffee mug
26

27. Inception network
(GoogLeNet)
• ImageNet classification challenge 2014
• 1000 object classes
• Top-5 accuracy 93.33%
• Trained on a few high-end GPUs within a week
• 1.2 million images for training
• Dumped model weights around 40MB
27

28. Inception network
(GoogLeNet)
Image source: http://joelouismarino.github.io/
Stem
9 inception modules
Output classifier
Helper classifiers
Convolution
Pooling
Softmax
Other
28

29. OpenCPU
• HTTP API for methods implemented in R packages
• Data scientists can provide useful methods
• Web developers can use them in various apps
29

30. Code
30 Image source: www.newscientist.com

31. Possible applications
• Monitoring systems
• Ordering food when fridge is empty
• Tagging photos
• Web crawling
31

32. Possible improvements
• Use of Natural Language Processing
• Using multiple cameras
• Run on Raspberry Pi
• Use ML platform
• Microsoft Computer Vision API
• Google Vision API
• Amazon Rekognition
32

33. Questions?