SlideShare a Scribd company logo
Teaching Machines to See
The process of designing (datasets) with AI
Ryan Compton
ryan@clarifai.com
Clarifai
- Clarifai.com
- 40 people • $40M • Imagenet ‘13
- Image + Video Recognition
- NYC • SF
Current product applications
- Navigating and understanding internal image libraries
- Auto-proposing tags upon upload to a service
- ‘You may also like’ carousel
- Filtering user generated content
Framework - mysterious but not magical
Demo
Easy to use API
How simple is it to Search?
How simple is it to Train?
So why is it so easy ?
Representation learning
Learned
Representations
Learned
Representations
Without convnets?
Object detection in 1978
Object detection in 1996
Jain, A. K., Zhong, Y., & Lakshmanan, S. (1996). Object matching using deformable templates. IEEE Transactions on pattern
analysis and machine intelligence, 18(3), 267-278.
Learning templates in 2009
https://arxiv.org/abs/1104.2123
Disclaimer: This presentation will contain visualizations of
nudity for scientific purposes.
1996
Eventually someone will get real good at feature
engineering and the problem will be solved, right?
Fast forward: CNNs
But how ?
SFW training data
NSFW training data
convnet
Demo network for this talk (modified ZF net):
5 convolutional layers
3 fully-connected layers
2 softmax outputs (‘sfw’ and ‘nsfw’)
69,961,504 parameters
Train Test
Train Test
Publish
Train Test
Deploy to prod
and everything
breaks
And how does it do that ?
Occlusion
Experiment
Occlusion
Experiment
Occlusion
Experiment
Deconvolutional Networks
• Provides way to map activations at high
layers back to the input
• Same operations as Convnet, but in reverse:
– Unpool feature maps
– Convolve unpooled maps
• Filters copied from Convnet
– Used here purely as a probe
– Originally proposed as unsupervised learning
method
– No inference, no learning
– Compare with sparse coding Input Image
Convolution (learned)
Unpooling
Feature maps
Non-linearity
[Zeiler et al. CVPR’10, ICCV’11]
Sparse Coding
wavelets, curvelets, framelets, shapelets, shearlets...
Objective for a single layer in a
deconvnet
Reuse Feedforward Switches
Reversible Max Pooling
Pooled
Feature
Maps
Max
Locations
“Switches”
Pooling Unpooling
Feature Map
Reconstructed Feature Map
2014
First Layer
Image from Zeiler, Matthew D., and Rob Fergus. "Visualizing and understanding convolutional networks." European
Conference on Computer Vision. Springer International Publishing, 2014.
Second Layer
Third Layer
Fourth Layer
Fifth Layer
conv1
conv2
conv3
conv4
conv5
fc8
Oh and safe stuff too...
Oh and safe stuff too...
Progression during training
Progression during training
Progression during training
Progression during training
Progression during training
Recap
20161996
The Future...
Context matters
Context matters
Context matters
Context matters
Customization matters (demo)
Thank you
ryan@clarifai.com
DEVELOPERS
10,000
Free API Credits
Get started now!
developer.clarifai.com

More Related Content

Similar to Teaching machines to see the process of designing (datasets) with ai

What convnets look at when they look at nudity
What convnets look at when they look at nudityWhat convnets look at when they look at nudity
What convnets look at when they look at nudity
Ryan Compton
 
TechnicalBackgroundOverview
TechnicalBackgroundOverviewTechnicalBackgroundOverview
TechnicalBackgroundOverview
Motaz El-Saban
 
深度學習在AOI的應用
深度學習在AOI的應用深度學習在AOI的應用
深度學習在AOI的應用
CHENHuiMei
 
Introduction talk to Computer Vision
Introduction talk to Computer Vision Introduction talk to Computer Vision
Introduction talk to Computer Vision
Chen Sagiv
 
Convolutional Patch Representations for Image Retrieval An unsupervised approach
Convolutional Patch Representations for Image Retrieval An unsupervised approachConvolutional Patch Representations for Image Retrieval An unsupervised approach
Convolutional Patch Representations for Image Retrieval An unsupervised approach
Universitat de Barcelona
 
SaturdaysAI: CNN in Underwater Robotics and eHealth
SaturdaysAI: CNN in Underwater Robotics and eHealthSaturdaysAI: CNN in Underwater Robotics and eHealth
SaturdaysAI: CNN in Underwater Robotics and eHealth
Yolanda Gonzalez-Cid
 
Fa19_P1.pptx
Fa19_P1.pptxFa19_P1.pptx
Fa19_P1.pptx
Md Abul Hayat
 
Convolutional neural networks
Convolutional neural  networksConvolutional neural  networks
Convolutional neural networks
Learning Courses Online
 
Intro To Convolutional Neural Networks
Intro To Convolutional Neural NetworksIntro To Convolutional Neural Networks
Intro To Convolutional Neural Networks
Mark Scully
 
Motion and tracking
Motion and trackingMotion and tracking
Motion and tracking
potaters
 
Artificial Intelligence, Machine Learning and Deep Learning
Artificial Intelligence, Machine Learning and Deep LearningArtificial Intelligence, Machine Learning and Deep Learning
Artificial Intelligence, Machine Learning and Deep Learning
Sujit Pal
 
AI powered emotion recognition: From Inception to Production - Global AI Conf...
AI powered emotion recognition: From Inception to Production - Global AI Conf...AI powered emotion recognition: From Inception to Production - Global AI Conf...
AI powered emotion recognition: From Inception to Production - Global AI Conf...
Vandana Kannan
 
AI powered emotion recognition: From Inception to Production - Global AI Conf...
AI powered emotion recognition: From Inception to Production - Global AI Conf...AI powered emotion recognition: From Inception to Production - Global AI Conf...
AI powered emotion recognition: From Inception to Production - Global AI Conf...
Apache MXNet
 
Deep learning unsupervised learning diapo
Deep learning unsupervised learning diapoDeep learning unsupervised learning diapo
Deep learning unsupervised learning diapo
Milton Paja
 
IEEE Projects 2014-2015
IEEE Projects 2014-2015IEEE Projects 2014-2015
IEEE Projects 2014-2015
Finalyear Projects
 
Rapid object detection using boosted cascade of simple features
Rapid object detection using boosted  cascade of simple featuresRapid object detection using boosted  cascade of simple features
Rapid object detection using boosted cascade of simple features
Hirantha Pradeep
 
Lecture 29 Convolutional Neural Networks - Computer Vision Spring2015
Lecture 29 Convolutional Neural Networks -  Computer Vision Spring2015Lecture 29 Convolutional Neural Networks -  Computer Vision Spring2015
Lecture 29 Convolutional Neural Networks - Computer Vision Spring2015
Jia-Bin Huang
 
MLIP - Chapter 3 - Introduction to deep learning
MLIP - Chapter 3 - Introduction to deep learningMLIP - Chapter 3 - Introduction to deep learning
MLIP - Chapter 3 - Introduction to deep learning
Charles Deledalle
 
物件偵測與辨識技術
物件偵測與辨識技術物件偵測與辨識技術
物件偵測與辨識技術
CHENHuiMei
 
Development of Deep Learning Architecture
Development of Deep Learning ArchitectureDevelopment of Deep Learning Architecture
Development of Deep Learning Architecture
Pantech ProLabs India Pvt Ltd
 

Similar to Teaching machines to see the process of designing (datasets) with ai (20)

What convnets look at when they look at nudity
What convnets look at when they look at nudityWhat convnets look at when they look at nudity
What convnets look at when they look at nudity
 
TechnicalBackgroundOverview
TechnicalBackgroundOverviewTechnicalBackgroundOverview
TechnicalBackgroundOverview
 
深度學習在AOI的應用
深度學習在AOI的應用深度學習在AOI的應用
深度學習在AOI的應用
 
Introduction talk to Computer Vision
Introduction talk to Computer Vision Introduction talk to Computer Vision
Introduction talk to Computer Vision
 
Convolutional Patch Representations for Image Retrieval An unsupervised approach
Convolutional Patch Representations for Image Retrieval An unsupervised approachConvolutional Patch Representations for Image Retrieval An unsupervised approach
Convolutional Patch Representations for Image Retrieval An unsupervised approach
 
SaturdaysAI: CNN in Underwater Robotics and eHealth
SaturdaysAI: CNN in Underwater Robotics and eHealthSaturdaysAI: CNN in Underwater Robotics and eHealth
SaturdaysAI: CNN in Underwater Robotics and eHealth
 
Fa19_P1.pptx
Fa19_P1.pptxFa19_P1.pptx
Fa19_P1.pptx
 
Convolutional neural networks
Convolutional neural  networksConvolutional neural  networks
Convolutional neural networks
 
Intro To Convolutional Neural Networks
Intro To Convolutional Neural NetworksIntro To Convolutional Neural Networks
Intro To Convolutional Neural Networks
 
Motion and tracking
Motion and trackingMotion and tracking
Motion and tracking
 
Artificial Intelligence, Machine Learning and Deep Learning
Artificial Intelligence, Machine Learning and Deep LearningArtificial Intelligence, Machine Learning and Deep Learning
Artificial Intelligence, Machine Learning and Deep Learning
 
AI powered emotion recognition: From Inception to Production - Global AI Conf...
AI powered emotion recognition: From Inception to Production - Global AI Conf...AI powered emotion recognition: From Inception to Production - Global AI Conf...
AI powered emotion recognition: From Inception to Production - Global AI Conf...
 
AI powered emotion recognition: From Inception to Production - Global AI Conf...
AI powered emotion recognition: From Inception to Production - Global AI Conf...AI powered emotion recognition: From Inception to Production - Global AI Conf...
AI powered emotion recognition: From Inception to Production - Global AI Conf...
 
Deep learning unsupervised learning diapo
Deep learning unsupervised learning diapoDeep learning unsupervised learning diapo
Deep learning unsupervised learning diapo
 
IEEE Projects 2014-2015
IEEE Projects 2014-2015IEEE Projects 2014-2015
IEEE Projects 2014-2015
 
Rapid object detection using boosted cascade of simple features
Rapid object detection using boosted  cascade of simple featuresRapid object detection using boosted  cascade of simple features
Rapid object detection using boosted cascade of simple features
 
Lecture 29 Convolutional Neural Networks - Computer Vision Spring2015
Lecture 29 Convolutional Neural Networks -  Computer Vision Spring2015Lecture 29 Convolutional Neural Networks -  Computer Vision Spring2015
Lecture 29 Convolutional Neural Networks - Computer Vision Spring2015
 
MLIP - Chapter 3 - Introduction to deep learning
MLIP - Chapter 3 - Introduction to deep learningMLIP - Chapter 3 - Introduction to deep learning
MLIP - Chapter 3 - Introduction to deep learning
 
物件偵測與辨識技術
物件偵測與辨識技術物件偵測與辨識技術
物件偵測與辨識技術
 
Development of Deep Learning Architecture
Development of Deep Learning ArchitectureDevelopment of Deep Learning Architecture
Development of Deep Learning Architecture
 

More from DevFest DC

Push Notifications Or: How I Learned to Stop Worrying and Love NotificationCo...
Push Notifications Or: How I Learned to Stop Worrying and Love NotificationCo...Push Notifications Or: How I Learned to Stop Worrying and Love NotificationCo...
Push Notifications Or: How I Learned to Stop Worrying and Love NotificationCo...
DevFest DC
 
Reactive Programming in Akka
Reactive Programming in AkkaReactive Programming in Akka
Reactive Programming in Akka
DevFest DC
 
Containers, microservices and azure
Containers, microservices and azureContainers, microservices and azure
Containers, microservices and azure
DevFest DC
 
Programming Google apps with the G Suite APIs
Programming Google apps with the G Suite APIsProgramming Google apps with the G Suite APIs
Programming Google apps with the G Suite APIs
DevFest DC
 
Snowflakes in the Cloud Real world experience on a new approach for Big Data
Snowflakes in the Cloud Real world experience on a new approach for Big DataSnowflakes in the Cloud Real world experience on a new approach for Big Data
Snowflakes in the Cloud Real world experience on a new approach for Big Data
DevFest DC
 
Well, That Escalated Quickly: Anomaly Detection with Elastic Machine Learning
Well, That Escalated Quickly: Anomaly Detection with Elastic Machine LearningWell, That Escalated Quickly: Anomaly Detection with Elastic Machine Learning
Well, That Escalated Quickly: Anomaly Detection with Elastic Machine Learning
DevFest DC
 
Why uri storage and the modern android app
Why uri  storage and the modern android appWhy uri  storage and the modern android app
Why uri storage and the modern android app
DevFest DC
 
Myths of Angular 2: What Angular Really Is
Myths of Angular 2: What Angular Really IsMyths of Angular 2: What Angular Really Is
Myths of Angular 2: What Angular Really Is
DevFest DC
 
Android Things Robocar with TensorFlow for object recognition
Android Things Robocar with TensorFlow for object recognitionAndroid Things Robocar with TensorFlow for object recognition
Android Things Robocar with TensorFlow for object recognition
DevFest DC
 
Troubleshooting & debugging production microservices in Kubernetes with Googl...
Troubleshooting & debugging production microservices in Kubernetes with Googl...Troubleshooting & debugging production microservices in Kubernetes with Googl...
Troubleshooting & debugging production microservices in Kubernetes with Googl...
DevFest DC
 
Hack the Real World with ANDROID THINGS
Hack the Real World with ANDROID THINGSHack the Real World with ANDROID THINGS
Hack the Real World with ANDROID THINGS
DevFest DC
 
Using Cloud Vision To Watch The World’s News Imagery In Realtime: The GDELT P...
Using Cloud Vision To Watch The World’s News Imagery In Realtime: The GDELT P...Using Cloud Vision To Watch The World’s News Imagery In Realtime: The GDELT P...
Using Cloud Vision To Watch The World’s News Imagery In Realtime: The GDELT P...
DevFest DC
 

More from DevFest DC (12)

Push Notifications Or: How I Learned to Stop Worrying and Love NotificationCo...
Push Notifications Or: How I Learned to Stop Worrying and Love NotificationCo...Push Notifications Or: How I Learned to Stop Worrying and Love NotificationCo...
Push Notifications Or: How I Learned to Stop Worrying and Love NotificationCo...
 
Reactive Programming in Akka
Reactive Programming in AkkaReactive Programming in Akka
Reactive Programming in Akka
 
Containers, microservices and azure
Containers, microservices and azureContainers, microservices and azure
Containers, microservices and azure
 
Programming Google apps with the G Suite APIs
Programming Google apps with the G Suite APIsProgramming Google apps with the G Suite APIs
Programming Google apps with the G Suite APIs
 
Snowflakes in the Cloud Real world experience on a new approach for Big Data
Snowflakes in the Cloud Real world experience on a new approach for Big DataSnowflakes in the Cloud Real world experience on a new approach for Big Data
Snowflakes in the Cloud Real world experience on a new approach for Big Data
 
Well, That Escalated Quickly: Anomaly Detection with Elastic Machine Learning
Well, That Escalated Quickly: Anomaly Detection with Elastic Machine LearningWell, That Escalated Quickly: Anomaly Detection with Elastic Machine Learning
Well, That Escalated Quickly: Anomaly Detection with Elastic Machine Learning
 
Why uri storage and the modern android app
Why uri  storage and the modern android appWhy uri  storage and the modern android app
Why uri storage and the modern android app
 
Myths of Angular 2: What Angular Really Is
Myths of Angular 2: What Angular Really IsMyths of Angular 2: What Angular Really Is
Myths of Angular 2: What Angular Really Is
 
Android Things Robocar with TensorFlow for object recognition
Android Things Robocar with TensorFlow for object recognitionAndroid Things Robocar with TensorFlow for object recognition
Android Things Robocar with TensorFlow for object recognition
 
Troubleshooting & debugging production microservices in Kubernetes with Googl...
Troubleshooting & debugging production microservices in Kubernetes with Googl...Troubleshooting & debugging production microservices in Kubernetes with Googl...
Troubleshooting & debugging production microservices in Kubernetes with Googl...
 
Hack the Real World with ANDROID THINGS
Hack the Real World with ANDROID THINGSHack the Real World with ANDROID THINGS
Hack the Real World with ANDROID THINGS
 
Using Cloud Vision To Watch The World’s News Imagery In Realtime: The GDELT P...
Using Cloud Vision To Watch The World’s News Imagery In Realtime: The GDELT P...Using Cloud Vision To Watch The World’s News Imagery In Realtime: The GDELT P...
Using Cloud Vision To Watch The World’s News Imagery In Realtime: The GDELT P...
 

Recently uploaded

What Challenges Await Beginners in Snowshoeing
What Challenges Await Beginners in SnowshoeingWhat Challenges Await Beginners in Snowshoeing
What Challenges Await Beginners in Snowshoeing
Snowshoe Tahoe
 
Excursions in Tahiti Island Adventure
Excursions in Tahiti Island AdventureExcursions in Tahiti Island Adventure
Excursions in Tahiti Island Adventure
Unique Tahiti
 
Wayanad-The-Touristry-Heaven to the tour.pptx
Wayanad-The-Touristry-Heaven to the tour.pptxWayanad-The-Touristry-Heaven to the tour.pptx
Wayanad-The-Touristry-Heaven to the tour.pptx
cosmo-soil
 
Discovering Egypt A Step-by-Step Guide to Planning Your Trip.ppt
Discovering Egypt A Step-by-Step Guide to Planning Your Trip.pptDiscovering Egypt A Step-by-Step Guide to Planning Your Trip.ppt
Discovering Egypt A Step-by-Step Guide to Planning Your Trip.ppt
Imperial Egypt
 
Assessing the Influence of Transportation on the Tourism Industry in Nigeria
Assessing the Influence of Transportation on the  Tourism Industry in NigeriaAssessing the Influence of Transportation on the  Tourism Industry in Nigeria
Assessing the Influence of Transportation on the Tourism Industry in Nigeria
gsochially
 
Understanding Bus Hire ServicesIN MELBOURNE .pptx
Understanding Bus Hire ServicesIN MELBOURNE .pptxUnderstanding Bus Hire ServicesIN MELBOURNE .pptx
Understanding Bus Hire ServicesIN MELBOURNE .pptx
MELBOURNEBUSHIRE
 
在线办理(BU毕业证书)波士顿大学毕业证录取通知书一模一样
在线办理(BU毕业证书)波士顿大学毕业证录取通知书一模一样在线办理(BU毕业证书)波士顿大学毕业证录取通知书一模一样
在线办理(BU毕业证书)波士顿大学毕业证录取通知书一模一样
v6ldcxuq
 
Un viaje a Buenos Aires y sus alrededores
Un viaje a Buenos Aires y sus alrededoresUn viaje a Buenos Aires y sus alrededores
Un viaje a Buenos Aires y sus alrededores
Judy Hochberg
 
Agama Tours&Safaris-Kilimanjaro day trip
Agama Tours&Safaris-Kilimanjaro day tripAgama Tours&Safaris-Kilimanjaro day trip
Agama Tours&Safaris-Kilimanjaro day trip
Agama Tours
 
How To Change Your Name On American Airlines Aadvantage.pptx
How To Change Your Name On American Airlines Aadvantage.pptxHow To Change Your Name On American Airlines Aadvantage.pptx
How To Change Your Name On American Airlines Aadvantage.pptx
edqour001namechange
 
How Do I Plan a Kilimanjaro Climb? 7 Essential Tips Revealed.pdf
How Do I Plan a Kilimanjaro Climb? 7 Essential Tips Revealed.pdfHow Do I Plan a Kilimanjaro Climb? 7 Essential Tips Revealed.pdf
How Do I Plan a Kilimanjaro Climb? 7 Essential Tips Revealed.pdf
Eastafrica Travelcompany
 
How To Change A Name On American Airlines Ticket.pptx
How To Change A Name On American Airlines Ticket.pptxHow To Change A Name On American Airlines Ticket.pptx
How To Change A Name On American Airlines Ticket.pptx
edqour001namechange
 
Best Places to Stay in New Brunswick, Canada.
Best Places to Stay in New Brunswick, Canada.Best Places to Stay in New Brunswick, Canada.
Best Places to Stay in New Brunswick, Canada.
Mahogany Manor
 
Top 10 Tourist Places in South India to Explore.pdf
Top 10 Tourist Places in South India to Explore.pdfTop 10 Tourist Places in South India to Explore.pdf
Top 10 Tourist Places in South India to Explore.pdf
Savita Yadav
 
Un viaje a Argentina updated xxxxxxxxxxx
Un viaje a Argentina updated xxxxxxxxxxxUn viaje a Argentina updated xxxxxxxxxxx
Un viaje a Argentina updated xxxxxxxxxxx
Judy Hochberg
 

Recently uploaded (15)

What Challenges Await Beginners in Snowshoeing
What Challenges Await Beginners in SnowshoeingWhat Challenges Await Beginners in Snowshoeing
What Challenges Await Beginners in Snowshoeing
 
Excursions in Tahiti Island Adventure
Excursions in Tahiti Island AdventureExcursions in Tahiti Island Adventure
Excursions in Tahiti Island Adventure
 
Wayanad-The-Touristry-Heaven to the tour.pptx
Wayanad-The-Touristry-Heaven to the tour.pptxWayanad-The-Touristry-Heaven to the tour.pptx
Wayanad-The-Touristry-Heaven to the tour.pptx
 
Discovering Egypt A Step-by-Step Guide to Planning Your Trip.ppt
Discovering Egypt A Step-by-Step Guide to Planning Your Trip.pptDiscovering Egypt A Step-by-Step Guide to Planning Your Trip.ppt
Discovering Egypt A Step-by-Step Guide to Planning Your Trip.ppt
 
Assessing the Influence of Transportation on the Tourism Industry in Nigeria
Assessing the Influence of Transportation on the  Tourism Industry in NigeriaAssessing the Influence of Transportation on the  Tourism Industry in Nigeria
Assessing the Influence of Transportation on the Tourism Industry in Nigeria
 
Understanding Bus Hire ServicesIN MELBOURNE .pptx
Understanding Bus Hire ServicesIN MELBOURNE .pptxUnderstanding Bus Hire ServicesIN MELBOURNE .pptx
Understanding Bus Hire ServicesIN MELBOURNE .pptx
 
在线办理(BU毕业证书)波士顿大学毕业证录取通知书一模一样
在线办理(BU毕业证书)波士顿大学毕业证录取通知书一模一样在线办理(BU毕业证书)波士顿大学毕业证录取通知书一模一样
在线办理(BU毕业证书)波士顿大学毕业证录取通知书一模一样
 
Un viaje a Buenos Aires y sus alrededores
Un viaje a Buenos Aires y sus alrededoresUn viaje a Buenos Aires y sus alrededores
Un viaje a Buenos Aires y sus alrededores
 
Agama Tours&Safaris-Kilimanjaro day trip
Agama Tours&Safaris-Kilimanjaro day tripAgama Tours&Safaris-Kilimanjaro day trip
Agama Tours&Safaris-Kilimanjaro day trip
 
How To Change Your Name On American Airlines Aadvantage.pptx
How To Change Your Name On American Airlines Aadvantage.pptxHow To Change Your Name On American Airlines Aadvantage.pptx
How To Change Your Name On American Airlines Aadvantage.pptx
 
How Do I Plan a Kilimanjaro Climb? 7 Essential Tips Revealed.pdf
How Do I Plan a Kilimanjaro Climb? 7 Essential Tips Revealed.pdfHow Do I Plan a Kilimanjaro Climb? 7 Essential Tips Revealed.pdf
How Do I Plan a Kilimanjaro Climb? 7 Essential Tips Revealed.pdf
 
How To Change A Name On American Airlines Ticket.pptx
How To Change A Name On American Airlines Ticket.pptxHow To Change A Name On American Airlines Ticket.pptx
How To Change A Name On American Airlines Ticket.pptx
 
Best Places to Stay in New Brunswick, Canada.
Best Places to Stay in New Brunswick, Canada.Best Places to Stay in New Brunswick, Canada.
Best Places to Stay in New Brunswick, Canada.
 
Top 10 Tourist Places in South India to Explore.pdf
Top 10 Tourist Places in South India to Explore.pdfTop 10 Tourist Places in South India to Explore.pdf
Top 10 Tourist Places in South India to Explore.pdf
 
Un viaje a Argentina updated xxxxxxxxxxx
Un viaje a Argentina updated xxxxxxxxxxxUn viaje a Argentina updated xxxxxxxxxxx
Un viaje a Argentina updated xxxxxxxxxxx
 

Teaching machines to see the process of designing (datasets) with ai