SlideShare a Scribd company logo
1 of 36
Download to read offline
What Makes Computer
Vision Hard?
Anna Volokitin

voanna@vision.ee.ethz.ch

Geek Girls Carrots Zürich

23 January 2018
1
What is (computer)
vision?
2
converting pixels to
“meaning”
http://vision.stanford.edu/teaching/cs131_fall1617/lectures/lecture1_introduction_cs131_2016.pdf
3
Image Recognition
4
Segmentation
5
Action Recognition
6
Ideally, we want to automatically
answer any question about a
photo or video.
7
e.g. why is he smiling?
http://i.dailymail.co.uk/i/pix/2012/12/20/article-2250728-1696C1CD000005DC-817_964x628.jpg8
Returning to the basic
task of recognition — is it
really that hard?
9
Block world
• We can write rules for
recognising objects if we have 

• perfect lighting

• simple shapes, 

• etc

• Example: triangular prism

• Step 1) Find image edges

• Step 2) If any lines make a
triangle, we found it.
10
What about the real
world?
11
Viewpoint
Michelangelo 1475-1564
12
Illumination
slide credit: S. Ullman
13
Occlusion
Magritte, 1957
14
Deformation
Xu, Beihong 194315
Background Clutter
Klimt, 1913
16
Intra-class variation
17
Rule based traditional
programming won’t work
- there are too many rules
18
Have to learn to
recognise objects
19
Supervised Learning
training set
test set
20
Supervised Learning
• A model is way to decide whether an image is a cat or a
dog, based on some parameter

• In this case, parameter is average colour

• Training a model = adjusting parameter to make model
more accurate
is the image mainly this color ?
Yes
No
CAT
DOG
21
Training
is the image mainly this color ?
Yes
No
CAT
DOG
training set
Model outputs that all cat images are DOG,
so 50% accuracy22
Training
is the image mainly this color ?
Yes
No
CAT
DOG
training set
Model outputs that all cat images are DOG,
so 50% accuracy23
Training
is the image mainly this color ?
Yes
No
CAT
DOG
training set
Model outputs that all cat images are DOG,
so 50% accuracy24
Training
is the image mainly this color ?
Yes
No
CAT
DOG
training set
Model outputs that all cat images are DOG,
so 50% accuracy25
Training
is the image mainly this color ?
Yes
No
CAT
DOG
training set
100% accuracy, finally. Done training.26
Testing
100% test accuracy, great!
27
is the image mainly this color ?
Yes
No
CAT
DOG
But what if…
FAIL!
28
is the image mainly this color ?
Yes
No
CAT
DOG
We need more parameters
that we can tune to learn
complicated rules
29
Convolutional Neural
Networks
30
Demo
http://playground.tensorflow.org/
31
What do neural networks
learn?
32
So what can we do
with CNNs?
33
Demo
http://places2.csail.mit.edu/demo.html
34
35
Summary
• To quantify meaning, CV breaks problems down into
recognition, segmentation, etc.

• Hard because of the huge variability of image appearance

• Supervised learning to discover rules

• Lots of cool applications with CNNs

• recognition

• segmentation

• style transfer …
36

More Related Content

Similar to What Makes Computer Vision Hard?

Git334 Week 1 Lecture
Git334 Week 1 LectureGit334 Week 1 Lecture
Git334 Week 1 Lecturechadwestover
 
Laureate Online Education Internet and Multimedia Technolog.docx
Laureate Online Education    Internet and Multimedia Technolog.docxLaureate Online Education    Internet and Multimedia Technolog.docx
Laureate Online Education Internet and Multimedia Technolog.docxDIPESH30
 
Convolutional Neural Networks for Computer vision Applications
Convolutional Neural Networks for Computer vision ApplicationsConvolutional Neural Networks for Computer vision Applications
Convolutional Neural Networks for Computer vision ApplicationsAlex Conway
 
3D Animation Process and Workflow
3D Animation Process and Workflow3D Animation Process and Workflow
3D Animation Process and WorkflowGridway Digital
 
introduction to Digital Image Processing
introduction to Digital Image Processingintroduction to Digital Image Processing
introduction to Digital Image Processingnikesh gadare
 
Digital Techniques
Digital TechniquesDigital Techniques
Digital Techniquesvcraig
 
How to implement artificial intelligence solutions
How to implement artificial intelligence solutionsHow to implement artificial intelligence solutions
How to implement artificial intelligence solutionsCarlos Toxtli
 
Deep Learning for Computer Vision - ExecutiveML
Deep Learning for Computer Vision - ExecutiveMLDeep Learning for Computer Vision - ExecutiveML
Deep Learning for Computer Vision - ExecutiveMLAlex Conway
 
“Practical Image Data Augmentation Methods for Training Deep Learning Object ...
“Practical Image Data Augmentation Methods for Training Deep Learning Object ...“Practical Image Data Augmentation Methods for Training Deep Learning Object ...
“Practical Image Data Augmentation Methods for Training Deep Learning Object ...Edge AI and Vision Alliance
 
Week2- Deep Learning Intuition.pptx
Week2- Deep Learning Intuition.pptxWeek2- Deep Learning Intuition.pptx
Week2- Deep Learning Intuition.pptxfahmi324663
 
When indexes are not enough
When indexes are not enoughWhen indexes are not enough
When indexes are not enoughDavide Mauri
 
From Image Processing To Computer Vision
From Image Processing To Computer VisionFrom Image Processing To Computer Vision
From Image Processing To Computer VisionJoud Khattab
 
Mathematics in everyday life
Mathematics in everyday lifeMathematics in everyday life
Mathematics in everyday lifePrathika Jp Jp
 
DN18 | Demystifying the Buzz in Machine Learning! (This Time for Real) | Dat ...
DN18 | Demystifying the Buzz in Machine Learning! (This Time for Real) | Dat ...DN18 | Demystifying the Buzz in Machine Learning! (This Time for Real) | Dat ...
DN18 | Demystifying the Buzz in Machine Learning! (This Time for Real) | Dat ...Dataconomy Media
 
“Vision-language Representations for Robotics,” a Presentation from the Unive...
“Vision-language Representations for Robotics,” a Presentation from the Unive...“Vision-language Representations for Robotics,” a Presentation from the Unive...
“Vision-language Representations for Robotics,” a Presentation from the Unive...Edge AI and Vision Alliance
 
People detection in a video
People detection in a videoPeople detection in a video
People detection in a videoYonatan Katz
 
Describe Machine learning with math.
Describe Machine learning with math.Describe Machine learning with math.
Describe Machine learning with math.Takayuki Sawada
 

Similar to What Makes Computer Vision Hard? (18)

Git334 Week 1 Lecture
Git334 Week 1 LectureGit334 Week 1 Lecture
Git334 Week 1 Lecture
 
Laureate Online Education Internet and Multimedia Technolog.docx
Laureate Online Education    Internet and Multimedia Technolog.docxLaureate Online Education    Internet and Multimedia Technolog.docx
Laureate Online Education Internet and Multimedia Technolog.docx
 
Convolutional Neural Networks for Computer vision Applications
Convolutional Neural Networks for Computer vision ApplicationsConvolutional Neural Networks for Computer vision Applications
Convolutional Neural Networks for Computer vision Applications
 
3D Animation Process and Workflow
3D Animation Process and Workflow3D Animation Process and Workflow
3D Animation Process and Workflow
 
introduction to Digital Image Processing
introduction to Digital Image Processingintroduction to Digital Image Processing
introduction to Digital Image Processing
 
Bachelor's Project.pdf
Bachelor's Project.pdfBachelor's Project.pdf
Bachelor's Project.pdf
 
Digital Techniques
Digital TechniquesDigital Techniques
Digital Techniques
 
How to implement artificial intelligence solutions
How to implement artificial intelligence solutionsHow to implement artificial intelligence solutions
How to implement artificial intelligence solutions
 
Deep Learning for Computer Vision - ExecutiveML
Deep Learning for Computer Vision - ExecutiveMLDeep Learning for Computer Vision - ExecutiveML
Deep Learning for Computer Vision - ExecutiveML
 
“Practical Image Data Augmentation Methods for Training Deep Learning Object ...
“Practical Image Data Augmentation Methods for Training Deep Learning Object ...“Practical Image Data Augmentation Methods for Training Deep Learning Object ...
“Practical Image Data Augmentation Methods for Training Deep Learning Object ...
 
Week2- Deep Learning Intuition.pptx
Week2- Deep Learning Intuition.pptxWeek2- Deep Learning Intuition.pptx
Week2- Deep Learning Intuition.pptx
 
When indexes are not enough
When indexes are not enoughWhen indexes are not enough
When indexes are not enough
 
From Image Processing To Computer Vision
From Image Processing To Computer VisionFrom Image Processing To Computer Vision
From Image Processing To Computer Vision
 
Mathematics in everyday life
Mathematics in everyday lifeMathematics in everyday life
Mathematics in everyday life
 
DN18 | Demystifying the Buzz in Machine Learning! (This Time for Real) | Dat ...
DN18 | Demystifying the Buzz in Machine Learning! (This Time for Real) | Dat ...DN18 | Demystifying the Buzz in Machine Learning! (This Time for Real) | Dat ...
DN18 | Demystifying the Buzz in Machine Learning! (This Time for Real) | Dat ...
 
“Vision-language Representations for Robotics,” a Presentation from the Unive...
“Vision-language Representations for Robotics,” a Presentation from the Unive...“Vision-language Representations for Robotics,” a Presentation from the Unive...
“Vision-language Representations for Robotics,” a Presentation from the Unive...
 
People detection in a video
People detection in a videoPeople detection in a video
People detection in a video
 
Describe Machine learning with math.
Describe Machine learning with math.Describe Machine learning with math.
Describe Machine learning with math.
 

More from Geek Girls Carrots Switzerland

More from Geek Girls Carrots Switzerland (7)

Machine Learning For Personalized Fertility Predictions
Machine Learning For Personalized Fertility PredictionsMachine Learning For Personalized Fertility Predictions
Machine Learning For Personalized Fertility Predictions
 
Margarita Chli about TEACHING ROBOTS TO SEE
Margarita Chli about TEACHING ROBOTS TO SEEMargarita Chli about TEACHING ROBOTS TO SEE
Margarita Chli about TEACHING ROBOTS TO SEE
 
JAHLELA HASLE about HOW TO GET INTO IT
JAHLELA HASLE about HOW TO GET INTO ITJAHLELA HASLE about HOW TO GET INTO IT
JAHLELA HASLE about HOW TO GET INTO IT
 
Iwona Fluda: “Thinking in SEO language.”
Iwona Fluda: “Thinking in SEO language.”Iwona Fluda: “Thinking in SEO language.”
Iwona Fluda: “Thinking in SEO language.”
 
Wireless Technologies For The Internet Of Things
Wireless Technologies For The Internet Of ThingsWireless Technologies For The Internet Of Things
Wireless Technologies For The Internet Of Things
 
Artificial Intelligence in Personalized Health
Artificial Intelligence in Personalized HealthArtificial Intelligence in Personalized Health
Artificial Intelligence in Personalized Health
 
Dynamic Information Design
Dynamic Information DesignDynamic Information Design
Dynamic Information Design
 

Recently uploaded

🐬 The future of MySQL is Postgres 🐘
🐬  The future of MySQL is Postgres   🐘🐬  The future of MySQL is Postgres   🐘
🐬 The future of MySQL is Postgres 🐘RTylerCroy
 
How to Troubleshoot Apps for the Modern Connected Worker
How to Troubleshoot Apps for the Modern Connected WorkerHow to Troubleshoot Apps for the Modern Connected Worker
How to Troubleshoot Apps for the Modern Connected WorkerThousandEyes
 
presentation ICT roal in 21st century education
presentation ICT roal in 21st century educationpresentation ICT roal in 21st century education
presentation ICT roal in 21st century educationjfdjdjcjdnsjd
 
A Domino Admins Adventures (Engage 2024)
A Domino Admins Adventures (Engage 2024)A Domino Admins Adventures (Engage 2024)
A Domino Admins Adventures (Engage 2024)Gabriella Davis
 
Histor y of HAM Radio presentation slide
Histor y of HAM Radio presentation slideHistor y of HAM Radio presentation slide
Histor y of HAM Radio presentation slidevu2urc
 
Scaling API-first – The story of a global engineering organization
Scaling API-first – The story of a global engineering organizationScaling API-first – The story of a global engineering organization
Scaling API-first – The story of a global engineering organizationRadu Cotescu
 
2024: Domino Containers - The Next Step. News from the Domino Container commu...
2024: Domino Containers - The Next Step. News from the Domino Container commu...2024: Domino Containers - The Next Step. News from the Domino Container commu...
2024: Domino Containers - The Next Step. News from the Domino Container commu...Martijn de Jong
 
GenAI Risks & Security Meetup 01052024.pdf
GenAI Risks & Security Meetup 01052024.pdfGenAI Risks & Security Meetup 01052024.pdf
GenAI Risks & Security Meetup 01052024.pdflior mazor
 
Mastering MySQL Database Architecture: Deep Dive into MySQL Shell and MySQL R...
Mastering MySQL Database Architecture: Deep Dive into MySQL Shell and MySQL R...Mastering MySQL Database Architecture: Deep Dive into MySQL Shell and MySQL R...
Mastering MySQL Database Architecture: Deep Dive into MySQL Shell and MySQL R...Miguel Araújo
 
Apidays New York 2024 - Scaling API-first by Ian Reasor and Radu Cotescu, Adobe
Apidays New York 2024 - Scaling API-first by Ian Reasor and Radu Cotescu, AdobeApidays New York 2024 - Scaling API-first by Ian Reasor and Radu Cotescu, Adobe
Apidays New York 2024 - Scaling API-first by Ian Reasor and Radu Cotescu, Adobeapidays
 
Strategize a Smooth Tenant-to-tenant Migration and Copilot Takeoff
Strategize a Smooth Tenant-to-tenant Migration and Copilot TakeoffStrategize a Smooth Tenant-to-tenant Migration and Copilot Takeoff
Strategize a Smooth Tenant-to-tenant Migration and Copilot Takeoffsammart93
 
Axa Assurance Maroc - Insurer Innovation Award 2024
Axa Assurance Maroc - Insurer Innovation Award 2024Axa Assurance Maroc - Insurer Innovation Award 2024
Axa Assurance Maroc - Insurer Innovation Award 2024The Digital Insurer
 
Boost PC performance: How more available memory can improve productivity
Boost PC performance: How more available memory can improve productivityBoost PC performance: How more available memory can improve productivity
Boost PC performance: How more available memory can improve productivityPrincipled Technologies
 
AWS Community Day CPH - Three problems of Terraform
AWS Community Day CPH - Three problems of TerraformAWS Community Day CPH - Three problems of Terraform
AWS Community Day CPH - Three problems of TerraformAndrey Devyatkin
 
Handwritten Text Recognition for manuscripts and early printed texts
Handwritten Text Recognition for manuscripts and early printed textsHandwritten Text Recognition for manuscripts and early printed texts
Handwritten Text Recognition for manuscripts and early printed textsMaria Levchenko
 
Driving Behavioral Change for Information Management through Data-Driven Gree...
Driving Behavioral Change for Information Management through Data-Driven Gree...Driving Behavioral Change for Information Management through Data-Driven Gree...
Driving Behavioral Change for Information Management through Data-Driven Gree...Enterprise Knowledge
 
TrustArc Webinar - Stay Ahead of US State Data Privacy Law Developments
TrustArc Webinar - Stay Ahead of US State Data Privacy Law DevelopmentsTrustArc Webinar - Stay Ahead of US State Data Privacy Law Developments
TrustArc Webinar - Stay Ahead of US State Data Privacy Law DevelopmentsTrustArc
 
TrustArc Webinar - Unlock the Power of AI-Driven Data Discovery
TrustArc Webinar - Unlock the Power of AI-Driven Data DiscoveryTrustArc Webinar - Unlock the Power of AI-Driven Data Discovery
TrustArc Webinar - Unlock the Power of AI-Driven Data DiscoveryTrustArc
 
What Are The Drone Anti-jamming Systems Technology?
What Are The Drone Anti-jamming Systems Technology?What Are The Drone Anti-jamming Systems Technology?
What Are The Drone Anti-jamming Systems Technology?Antenna Manufacturer Coco
 
From Event to Action: Accelerate Your Decision Making with Real-Time Automation
From Event to Action: Accelerate Your Decision Making with Real-Time AutomationFrom Event to Action: Accelerate Your Decision Making with Real-Time Automation
From Event to Action: Accelerate Your Decision Making with Real-Time AutomationSafe Software
 

Recently uploaded (20)

🐬 The future of MySQL is Postgres 🐘
🐬  The future of MySQL is Postgres   🐘🐬  The future of MySQL is Postgres   🐘
🐬 The future of MySQL is Postgres 🐘
 
How to Troubleshoot Apps for the Modern Connected Worker
How to Troubleshoot Apps for the Modern Connected WorkerHow to Troubleshoot Apps for the Modern Connected Worker
How to Troubleshoot Apps for the Modern Connected Worker
 
presentation ICT roal in 21st century education
presentation ICT roal in 21st century educationpresentation ICT roal in 21st century education
presentation ICT roal in 21st century education
 
A Domino Admins Adventures (Engage 2024)
A Domino Admins Adventures (Engage 2024)A Domino Admins Adventures (Engage 2024)
A Domino Admins Adventures (Engage 2024)
 
Histor y of HAM Radio presentation slide
Histor y of HAM Radio presentation slideHistor y of HAM Radio presentation slide
Histor y of HAM Radio presentation slide
 
Scaling API-first – The story of a global engineering organization
Scaling API-first – The story of a global engineering organizationScaling API-first – The story of a global engineering organization
Scaling API-first – The story of a global engineering organization
 
2024: Domino Containers - The Next Step. News from the Domino Container commu...
2024: Domino Containers - The Next Step. News from the Domino Container commu...2024: Domino Containers - The Next Step. News from the Domino Container commu...
2024: Domino Containers - The Next Step. News from the Domino Container commu...
 
GenAI Risks & Security Meetup 01052024.pdf
GenAI Risks & Security Meetup 01052024.pdfGenAI Risks & Security Meetup 01052024.pdf
GenAI Risks & Security Meetup 01052024.pdf
 
Mastering MySQL Database Architecture: Deep Dive into MySQL Shell and MySQL R...
Mastering MySQL Database Architecture: Deep Dive into MySQL Shell and MySQL R...Mastering MySQL Database Architecture: Deep Dive into MySQL Shell and MySQL R...
Mastering MySQL Database Architecture: Deep Dive into MySQL Shell and MySQL R...
 
Apidays New York 2024 - Scaling API-first by Ian Reasor and Radu Cotescu, Adobe
Apidays New York 2024 - Scaling API-first by Ian Reasor and Radu Cotescu, AdobeApidays New York 2024 - Scaling API-first by Ian Reasor and Radu Cotescu, Adobe
Apidays New York 2024 - Scaling API-first by Ian Reasor and Radu Cotescu, Adobe
 
Strategize a Smooth Tenant-to-tenant Migration and Copilot Takeoff
Strategize a Smooth Tenant-to-tenant Migration and Copilot TakeoffStrategize a Smooth Tenant-to-tenant Migration and Copilot Takeoff
Strategize a Smooth Tenant-to-tenant Migration and Copilot Takeoff
 
Axa Assurance Maroc - Insurer Innovation Award 2024
Axa Assurance Maroc - Insurer Innovation Award 2024Axa Assurance Maroc - Insurer Innovation Award 2024
Axa Assurance Maroc - Insurer Innovation Award 2024
 
Boost PC performance: How more available memory can improve productivity
Boost PC performance: How more available memory can improve productivityBoost PC performance: How more available memory can improve productivity
Boost PC performance: How more available memory can improve productivity
 
AWS Community Day CPH - Three problems of Terraform
AWS Community Day CPH - Three problems of TerraformAWS Community Day CPH - Three problems of Terraform
AWS Community Day CPH - Three problems of Terraform
 
Handwritten Text Recognition for manuscripts and early printed texts
Handwritten Text Recognition for manuscripts and early printed textsHandwritten Text Recognition for manuscripts and early printed texts
Handwritten Text Recognition for manuscripts and early printed texts
 
Driving Behavioral Change for Information Management through Data-Driven Gree...
Driving Behavioral Change for Information Management through Data-Driven Gree...Driving Behavioral Change for Information Management through Data-Driven Gree...
Driving Behavioral Change for Information Management through Data-Driven Gree...
 
TrustArc Webinar - Stay Ahead of US State Data Privacy Law Developments
TrustArc Webinar - Stay Ahead of US State Data Privacy Law DevelopmentsTrustArc Webinar - Stay Ahead of US State Data Privacy Law Developments
TrustArc Webinar - Stay Ahead of US State Data Privacy Law Developments
 
TrustArc Webinar - Unlock the Power of AI-Driven Data Discovery
TrustArc Webinar - Unlock the Power of AI-Driven Data DiscoveryTrustArc Webinar - Unlock the Power of AI-Driven Data Discovery
TrustArc Webinar - Unlock the Power of AI-Driven Data Discovery
 
What Are The Drone Anti-jamming Systems Technology?
What Are The Drone Anti-jamming Systems Technology?What Are The Drone Anti-jamming Systems Technology?
What Are The Drone Anti-jamming Systems Technology?
 
From Event to Action: Accelerate Your Decision Making with Real-Time Automation
From Event to Action: Accelerate Your Decision Making with Real-Time AutomationFrom Event to Action: Accelerate Your Decision Making with Real-Time Automation
From Event to Action: Accelerate Your Decision Making with Real-Time Automation
 

What Makes Computer Vision Hard?