Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.
Brand recognition in real-life photos
Łukasz Czarnecki
• Machine Learning Engineer /Data Scientist
(Samsung)
• Human Tech Art founder and organiser
Łukasz Czarnecki
Monthly Actives Likes Daily Average Photos
Per Day
300M
Photos shared
30B+ 2.5B 70M
Instagram
Twitter
Facebook
Instagram
0...
Feed noise
What’s seen by Instagram?
Brand recognition in real-life photos
Social media engagement
Deep Learning
Deep Learning
Needs big
amount of
examples
Great results
in Image
Recognition
Hard to train
Learns
features
VS VS
heavy data
augmentation
&
train your
network
Use pretrained
network
&
SVM
Use pretrained
network
&
adjust last
layer...
Sliding window
• 16 windows: width/3
• 9 windows: width/2
• 1 window: width
NUMERIC GALLERY SAMPLE
406 420351
340 255301
Initial dataset
Adidas
Nike
Starbucks Heineken
COSTA COFFEE Carlsberg
Random class
Network propagation time
Testing different
networks
VGG_S Network
Optimizing
the code
Switched for
batch processing
Changi...
Network output
224 x 224 x 3
4096 x 1Network
How do you use the network?
How do you use that with SVM?
Testing results
Getting features
from Network
Training SVM
on features
Testing with
sliding
window
First results
68%Precission
66%Recall
Adding counterexamples
70%Precission
65%Recall
+2%
-1%
Using different layer
73%Precission
67%Recall
+3%
+1%
Adjusting probability cutoff
76%Precission
65%Recall
92%
+3%
-2%
Getting more data
82%Precission
68%Recall
1
2
3
4
5
Training with
initial dataset
Getting more
unlabeled data
Collecting w...
Data Augmentation
84%Precission
70%Recall
+2%
+2%
Adding normalisation
88%Precission
67%Recall
Getting features
from Network
Training SVM
on features
Testing with
sliding
w...
Examples of filtering
Final results
Adidas
Nike
Starbucks
COSTA COFFEE
Heineken
40%
47%
84%
80%
88%
80%
76%
93%
90%
97%
Precission Recall
Carlsb...
Tough examples
Thank you
Brand recognition in real-life photos using deep learning - Lukasz Czarnecki - PyData Berlin 2016
Brand recognition in real-life photos using deep learning - Lukasz Czarnecki - PyData Berlin 2016
Brand recognition in real-life photos using deep learning - Lukasz Czarnecki - PyData Berlin 2016
Brand recognition in real-life photos using deep learning - Lukasz Czarnecki - PyData Berlin 2016
Brand recognition in real-life photos using deep learning - Lukasz Czarnecki - PyData Berlin 2016
Brand recognition in real-life photos using deep learning - Lukasz Czarnecki - PyData Berlin 2016
Brand recognition in real-life photos using deep learning - Lukasz Czarnecki - PyData Berlin 2016
Brand recognition in real-life photos using deep learning - Lukasz Czarnecki - PyData Berlin 2016
Brand recognition in real-life photos using deep learning - Lukasz Czarnecki - PyData Berlin 2016
Brand recognition in real-life photos using deep learning - Lukasz Czarnecki - PyData Berlin 2016
Upcoming SlideShare
Loading in …5
×

Brand recognition in real-life photos using deep learning - Lukasz Czarnecki - PyData Berlin 2016

1,251 views

Published on

Project was created during a 3-months full-time data science program called Data Science Retreat. I’ve developed a brand recognition system that detects logos in real-life photos. It uses Deep Learning and SVM on Instagram images uploaded by users. It allows for better brand monitoring in social media independently from text descriptions and hashtags. I’ve used state-of-the-art Deep Learning tools (Theano, Lasagne, Caffe).

Published in: Technology
  • Be the first to comment

Brand recognition in real-life photos using deep learning - Lukasz Czarnecki - PyData Berlin 2016

  1. 1. Brand recognition in real-life photos Łukasz Czarnecki
  2. 2. • Machine Learning Engineer /Data Scientist (Samsung) • Human Tech Art founder and organiser Łukasz Czarnecki
  3. 3. Monthly Actives Likes Daily Average Photos Per Day 300M Photos shared 30B+ 2.5B 70M Instagram Twitter Facebook Instagram 0.03% 0.07% 4.21% Followers engagement on brand Source: Forrester Research, Inc.
  4. 4. Feed noise
  5. 5. What’s seen by Instagram?
  6. 6. Brand recognition in real-life photos
  7. 7. Social media engagement
  8. 8. Deep Learning Deep Learning Needs big amount of examples Great results in Image Recognition Hard to train Learns features
  9. 9. VS VS heavy data augmentation & train your network Use pretrained network & SVM Use pretrained network & adjust last layers Possible approaches
  10. 10. Sliding window • 16 windows: width/3 • 9 windows: width/2 • 1 window: width
  11. 11. NUMERIC GALLERY SAMPLE 406 420351 340 255301 Initial dataset Adidas Nike Starbucks Heineken COSTA COFFEE Carlsberg
  12. 12. Random class
  13. 13. Network propagation time Testing different networks VGG_S Network Optimizing the code Switched for batch processing Changing framework Caffe ~1 sec Initial time: 2 sec ~ 200 ms ~20 ms
  14. 14. Network output 224 x 224 x 3 4096 x 1Network
  15. 15. How do you use the network?
  16. 16. How do you use that with SVM?
  17. 17. Testing results
  18. 18. Getting features from Network Training SVM on features Testing with sliding window First results 68%Precission 66%Recall
  19. 19. Adding counterexamples 70%Precission 65%Recall +2% -1%
  20. 20. Using different layer 73%Precission 67%Recall +3% +1%
  21. 21. Adjusting probability cutoff 76%Precission 65%Recall 92% +3% -2%
  22. 22. Getting more data 82%Precission 68%Recall 1 2 3 4 5 Training with initial dataset Getting more unlabeled data Collecting windows with detections Manual segregation Training with bigger dataset +500 per class +6% +3%
  23. 23. Data Augmentation 84%Precission 70%Recall +2% +2%
  24. 24. Adding normalisation 88%Precission 67%Recall Getting features from Network Training SVM on features Testing with sliding window Normalisation +4% -3%
  25. 25. Examples of filtering
  26. 26. Final results Adidas Nike Starbucks COSTA COFFEE Heineken 40% 47% 84% 80% 88% 80% 76% 93% 90% 97% Precission Recall Carlsberg 66%93% 88%Precission 67%Recall
  27. 27. Tough examples
  28. 28. Thank you

×