The document discusses the evolution and application of auto-tagging technology for image recognition, emphasizing its usage across various businesses and sectors. It outlines the capabilities and features of tagging APIs, including the use of neural networks and deep learning for accurate image analysis and categorization. The document also presents practical use cases and upcoming features for enhancing image organization and recognition systems.

1. The Practical Evolution
of the Auto-Tagging
Technology as a Service
Georgi Kadrev @georgikadrev
Imagga Technologies @imagga
GTC ’17, May10th

2. more than
200Businesses
more than
8.9KDevelopers
in more than
82Countries Worldwide
In numbers

3. 3B
New photos shared
Every Day
more than

4. All the buzz words…
Artificial Intelligence Machine Learning
Deep Neural Network
Convolutional Neural NetworkDeep Learning

5. Why not
organize photos
using these
via an auto-tagging API?!

6. Image Recognition as Service
Submit an image
Get a list of tags or categories
Do whatever you need with them3
2
1

7. Frameworks Services
vs.

8. ‣ CudaConvnet
‣ Caffe
‣ Torch
‣ Theano
‣ MS Cognitive Toolkit
‣ TensorFlow
‣ Caffe2
Popular Frameworks

9. ‣ Collect data
‣ Train and optimize
‣ Deploy and handle scale
‣ Support and maintain
‣ Keep innovating
Frameworks
DIY (Do It Yourself)

10. ‣ Zero time to market
‣ No backend operations needed
‣ Pay as you go pricing
Services
Utilize a service

11. macaw
beak
parrot
bird
tropical wildlife
blue blacklight blue navy blue orange
10%
13%
100%
10%
100%
100%
Keyword and Color Tagging

12. 12
Beaches & Seaside
Sunrises & Sunsets
Nature & Landscape
Events & Parties
Category Tagging

13. NSFW
(Not Safe For Work)
‣ Safe
‣ Underwear
‣ Not safe
Content Moderation Tagging

14. Underlying Technology
Neural Networks
deep learning
+ feature extraction
+ semantic expansion
Automated Analysis
of raster/pixel data
Image Tagging
3,000+ objects
20,000+ terms/concepts

15. Traditional
Image Processing
and/or crowd-sourced
Deep Learning
Based
&

16. ‣ Amazon MT
‣ TagCow
‣ Imagga
‣ Imense
‣ CamFind (Cloudsight)
Traditional Image Processing

17. ‣ Imagga
‣ Clarifai
‣ Rekognition (Orbeus)
‣ Cloudsight
‣ Metamind
‣ MS Cognitive Services
‣ IBM Watson
‣ Google Cloud Vision
‣ Amazon Rekognition (acquired Orbeus)
‣ Salesforce Einstein (acquired Metamind)
Deep Learning Based

18. ‣ Detect faces
‣ Extract text
‣ Extract colors
‣ Analyze composition
‣ Classify/categorize scenes
‣ Recognize the main object
‣ Suggest multiple keywords
‣ Generate textual descriptions
‣ Recognize quality and art value
‣ Localize multiple objects
Practical Features

19. Detect Faces
Image source: Wikipedia, CC3 license

20. Extract Text
Image source: http://simon-tanner.blogspot.com/2015/06/text-capture-and-optical-character.html

21. Extract Colors
blue blacklight blue navy blue orange

22. Analyze Composition

23. 23
Beaches & Seaside
Sunrises & Sunsets
Nature & Landscape
Events & Parties
Classify/Categorize Scenes

24. Recognize The Main Object
parrot100%

25. Suggest Multiple Keywords
macaw
beak
parrot
bird
tropical wildlife 10%
13%
100%
10%
100%
100%

26. Localize Multiple Objects
Image source: https://www.linkedin.com/pulse/object-detection-using-deep-learning-advanced-users-part-1-sinhal

27. Generate Textual Descriptions
Image captured from: http://cs.stanford.edu/people/karpathy/deepimagesent/

28. Recognize Quality and Art Value
Image source: EyeEm Vision demo page

29. ‣ Internal organization of photos
‣ Organization of photos for sale
‣ Organization of personal photos
‣ Content moderation
‣ Marketing and advertising
‣ Analytics and profiling
‣ Content recommendation
Practical Use-cases

30. ‣ Cloud Services / Telcos
‣ Social Media Monitoring
‣ Contextual Advertising
‣ Digital Asset Management
‣ Image Processing Platforms
‣ Smart Devices and Installations
Use-cases

31. ‣ Is 95% of cloud storage photos and videos? YES!
‣ Does Google, Apple and Amazon have image recognition? YES!
‣ Do you have it? NO!
‣ Do you want to leave them ahead or you want to have it right now? NOW!
‣ What do you get? Increased cloud retention and lock in!
‣ Does it really work? YES!
‣ How do we know? Swisscom and 200+ more happy paying customers!
Use case: Cloud Service Providers/Telcos

32. Use-case: Swisscom
Enhancing Swisscom
myCloud with automated
image organization.
Tagging API
Technologies used:
Categorization API

33. Use-case: Unsplash
Unsplash
Providing powerful image
search capabilities.
Reduces/replace manual tagging and enhances
search.
Tagging API
Technologies used:

34. Use-case: Tavisca
Building a custom hotel
classifier.
Tagging API
Technologies used:
Custom Training
Automates classification and improving browsing
experience

35. Use-case: Tavisca
KIA K5 (Optima)
Creative Campaign
Very precise personalized targeting.
Tagging API
Technologies used:
Color Extraction API

36. Challenges
‣ Definition of the scope
‣ Data
‣ Even more data
‣ True learning from data
‣ Learning from private data

37. Upcoming Imagga features
‣ Positional object detection
‣ Official face recognition support
‣ Out-of-the-box on-premise packaging
‣ Official video support
‣ Logo and landmark recognition
‣ Better multi-language support

38. On-premise solution available
‣ World first
‣ Ultimate Privacy
‣ Enterprise class
‣ Easy deployment
‣ Custom categorization option
‣ Prepaid or pay-as-you go volume-based license
‣ Standard and advanced support per machine
On-premise
Solution
LAUNCHED
TODAY!

39. Integration & Business Model
On-premise
Solution
Cloud API
Platform
Monthly Subscription
Self-service
Volume License
S&M

40. Giants:
• Google Cloud Vision
• AWS Rekognition
• Microsoft Cognitive API
• IBM Watson Vision
• Salesforce Einstein
Imagga:
• On-premise option
• Professional custom training
Competitive Landscape
Notable Startups:
• Clarifai
• EyeEm (Vision)
• CloudSight

41. Thank You!
api@imagga.com twitter.com/imagga facebook.com/imagga