SlideShare a Scribd company logo
1 of 49
Lukas Biewald
The Effect of Better Algorithms
0%
5%
10%
15%
20%
25%
Naïve Bayes Maximum
Entropy
SVM
Classifier Error Rate
Active Semi-Supervised Learning for Improving
Word Alignment
(Vamshi ACL ’10)
Real World Data
2
The Effect of Better Features
0%
5%
10%
15%
20%
25%
30%
Unigrams Bigrams Unigrams+Bigrams
Classifier Error Rate
3
The Effect of More Data
Active Semi-Supervised Learning for
Improving Word Alignment
(Vamshi ACL ’10)
Real World Data
0%
2%
4%
6%
8%
10%
12%
14%
N 2N 4N
Classifier Error Rate
4
The Effect of Cleaner Data
0%
2%
4%
6%
8%
10%
12%
14%
90% Accurate Data 95% Accurate Data 100% Accurate Data
Classifier Error Rate
5
Where Do Data Scientists Spend Their Time?
6
Source: CrowdFlower Data
Science Report 2015
CrowdFlower Data Enrichment Platform
7
Color Data
8
9
10
11
12
13
14
Apple Watch
15
Apple Watch
16
Apple Watch
17
Apple Watch
18
Collecting the Same Data Over and Over
19
Open Data
20
Make Your Data Public Setting
21
Data for Everyone
22
Data For Everyone Library
23
Data for Everyone
24
Data For Everyone
25
Open Data API
26
URL Categorization
27
Categorize URLs
28
Record Data
29
Extracting Names and Titles
30
Summarization
31
Is an Image Funny?
32
Classifying Medical Images
33
Attributes of People
34
35
Kaggle accuracy
0%
10%
20%
30%
40%
50%
60%
70%
Baseline 12-May 13-May 14-May 15-May
Accuracy
Accuracy of Best Performing Model
36
Kaggle accuracy over time
0%
10%
20%
30%
40%
50%
60%
70%
80%
Accuracy
Accuracy of the Best Performing Model
37
Kaggle Participation
0
200
400
600
800
1000
1200
1400
Number of Participating Teams
38
AI
Classifier
Output
Confident
Human in the Loop
39
Human in the Loop
Confident
Output
AI
Classifier
Human
Annotation
40
Human in the Loop
Confident
Output
AI
Classifier
Active Learning
Human
Annotation
41
Human in the Loop
Confident
Output
AI
Classifier
Active Learning
Human
Annotation
42
Active Learning
From hunch.net active learning tutorial ICML ‘09
43
Active Learning Accuracy Improvement
44
Google Cars Miles Per Disengage
45
Adaptive Cruise Control
Image source: ExtremeTech
46
Advanced Chess
Image source: Computer Chess
47
AlphaGo
48
Lukas Biewald
lukas@crowdflower.com
@L2K
Thank You

More Related Content

Similar to Active Learning and Human-in-the-Loop

Cloud VM World Tradeshow Report
Cloud VM World Tradeshow ReportCloud VM World Tradeshow Report
Cloud VM World Tradeshow ReportCloud Cruiser, Inc
 
Managing Unrealistic User Expectations (i.e Digital for Spoilt Brats)
Managing Unrealistic User Expectations (i.e Digital for Spoilt Brats)Managing Unrealistic User Expectations (i.e Digital for Spoilt Brats)
Managing Unrealistic User Expectations (i.e Digital for Spoilt Brats)Dynatrace
 
Digital Disruption Asia - Pleasing the Unpleasable with Digital Performance Data
Digital Disruption Asia - Pleasing the Unpleasable with Digital Performance DataDigital Disruption Asia - Pleasing the Unpleasable with Digital Performance Data
Digital Disruption Asia - Pleasing the Unpleasable with Digital Performance DataDynatrace
 
How To Plan for, Design, and Implement a Communications Dashboard
How To Plan for, Design, and Implement a Communications DashboardHow To Plan for, Design, and Implement a Communications Dashboard
How To Plan for, Design, and Implement a Communications DashboardPaine Publishing
 
2020 Testing Trends: Top Predictions for QA Teams to Watch, Join, and Lead
2020 Testing Trends: Top Predictions for QA Teams to Watch, Join, and Lead2020 Testing Trends: Top Predictions for QA Teams to Watch, Join, and Lead
2020 Testing Trends: Top Predictions for QA Teams to Watch, Join, and LeadDevOps.com
 
Early Lessons Learned in Applying Big Data To TV Advertising
Early Lessons Learned in Applying Big Data To TV AdvertisingEarly Lessons Learned in Applying Big Data To TV Advertising
Early Lessons Learned in Applying Big Data To TV AdvertisingJeff Storan
 
Early Lessons Learned in Applying Big Data To TV Advertising
Early Lessons Learned in Applying Big Data To TV AdvertisingEarly Lessons Learned in Applying Big Data To TV Advertising
Early Lessons Learned in Applying Big Data To TV AdvertisingJeffrey Storan
 
Heavy, Messy, Misleading: How Big Data is a human problem, not a tech one
Heavy, Messy, Misleading: How Big Data is a human problem, not a tech oneHeavy, Messy, Misleading: How Big Data is a human problem, not a tech one
Heavy, Messy, Misleading: How Big Data is a human problem, not a tech onePulsar Platform
 
Population Stability Index(PSI) for Big Data World
Population Stability Index(PSI) for Big Data WorldPopulation Stability Index(PSI) for Big Data World
Population Stability Index(PSI) for Big Data WorldJeomoan Kurian
 
Digital disruption
Digital disruptionDigital disruption
Digital disruptionJerry Tan
 
JDO 2019: Data Science for Developers - Matthew Renze
JDO 2019: Data Science for Developers -  Matthew RenzeJDO 2019: Data Science for Developers -  Matthew Renze
JDO 2019: Data Science for Developers - Matthew RenzePROIDEA
 
FIAT/IFTA MAM Survey - Results overview
FIAT/IFTA MAM Survey - Results overviewFIAT/IFTA MAM Survey - Results overview
FIAT/IFTA MAM Survey - Results overviewBrecht Declercq
 
UX by the numbers: Discovering the why from numbers
UX by the numbers: Discovering the why from numbersUX by the numbers: Discovering the why from numbers
UX by the numbers: Discovering the why from numbersUXPA UK
 
Discovering WHY from numbers
Discovering WHY from numbersDiscovering WHY from numbers
Discovering WHY from numbersWebnographer
 
Measuring Video Effectiveness - Wistiafest 2015
Measuring Video Effectiveness - Wistiafest 2015Measuring Video Effectiveness - Wistiafest 2015
Measuring Video Effectiveness - Wistiafest 2015Brendan Schwartz
 
Pitch Deck Teardown: Lumigo's $29 million Series A deck
Pitch Deck Teardown: Lumigo's $29 million Series A deckPitch Deck Teardown: Lumigo's $29 million Series A deck
Pitch Deck Teardown: Lumigo's $29 million Series A deckHajeJanKamps
 
ITESOFT Insider View | The Importance of Capture
ITESOFT Insider View | The Importance of CaptureITESOFT Insider View | The Importance of Capture
ITESOFT Insider View | The Importance of CaptureITESOFT
 
2016 Yahoo Taiwan Mobile Developer Conference
2016 Yahoo Taiwan Mobile Developer Conference 2016 Yahoo Taiwan Mobile Developer Conference
2016 Yahoo Taiwan Mobile Developer Conference Flurry, Inc.
 

Similar to Active Learning and Human-in-the-Loop (20)

Cloud VM World Tradeshow Report
Cloud VM World Tradeshow ReportCloud VM World Tradeshow Report
Cloud VM World Tradeshow Report
 
Managing Unrealistic User Expectations (i.e Digital for Spoilt Brats)
Managing Unrealistic User Expectations (i.e Digital for Spoilt Brats)Managing Unrealistic User Expectations (i.e Digital for Spoilt Brats)
Managing Unrealistic User Expectations (i.e Digital for Spoilt Brats)
 
Digital Disruption Asia - Pleasing the Unpleasable with Digital Performance Data
Digital Disruption Asia - Pleasing the Unpleasable with Digital Performance DataDigital Disruption Asia - Pleasing the Unpleasable with Digital Performance Data
Digital Disruption Asia - Pleasing the Unpleasable with Digital Performance Data
 
The 2016 Watch List
The 2016 Watch ListThe 2016 Watch List
The 2016 Watch List
 
How To Plan for, Design, and Implement a Communications Dashboard
How To Plan for, Design, and Implement a Communications DashboardHow To Plan for, Design, and Implement a Communications Dashboard
How To Plan for, Design, and Implement a Communications Dashboard
 
2020 Testing Trends: Top Predictions for QA Teams to Watch, Join, and Lead
2020 Testing Trends: Top Predictions for QA Teams to Watch, Join, and Lead2020 Testing Trends: Top Predictions for QA Teams to Watch, Join, and Lead
2020 Testing Trends: Top Predictions for QA Teams to Watch, Join, and Lead
 
Early Lessons Learned in Applying Big Data To TV Advertising
Early Lessons Learned in Applying Big Data To TV AdvertisingEarly Lessons Learned in Applying Big Data To TV Advertising
Early Lessons Learned in Applying Big Data To TV Advertising
 
Early Lessons Learned in Applying Big Data To TV Advertising
Early Lessons Learned in Applying Big Data To TV AdvertisingEarly Lessons Learned in Applying Big Data To TV Advertising
Early Lessons Learned in Applying Big Data To TV Advertising
 
Heavy, Messy, Misleading: How Big Data is a human problem, not a tech one
Heavy, Messy, Misleading: How Big Data is a human problem, not a tech oneHeavy, Messy, Misleading: How Big Data is a human problem, not a tech one
Heavy, Messy, Misleading: How Big Data is a human problem, not a tech one
 
Population Stability Index(PSI) for Big Data World
Population Stability Index(PSI) for Big Data WorldPopulation Stability Index(PSI) for Big Data World
Population Stability Index(PSI) for Big Data World
 
Digital disruption
Digital disruptionDigital disruption
Digital disruption
 
JDO 2019: Data Science for Developers - Matthew Renze
JDO 2019: Data Science for Developers -  Matthew RenzeJDO 2019: Data Science for Developers -  Matthew Renze
JDO 2019: Data Science for Developers - Matthew Renze
 
FIAT/IFTA MAM Survey - Results overview
FIAT/IFTA MAM Survey - Results overviewFIAT/IFTA MAM Survey - Results overview
FIAT/IFTA MAM Survey - Results overview
 
UX by the numbers: Discovering the why from numbers
UX by the numbers: Discovering the why from numbersUX by the numbers: Discovering the why from numbers
UX by the numbers: Discovering the why from numbers
 
Discovering WHY from numbers
Discovering WHY from numbersDiscovering WHY from numbers
Discovering WHY from numbers
 
Measuring Video Effectiveness - Wistiafest 2015
Measuring Video Effectiveness - Wistiafest 2015Measuring Video Effectiveness - Wistiafest 2015
Measuring Video Effectiveness - Wistiafest 2015
 
[TestWarez 2017] Od testowania do monitoringu jakości – wyzwania Continuous ...
[TestWarez 2017]  Od testowania do monitoringu jakości – wyzwania Continuous ...[TestWarez 2017]  Od testowania do monitoringu jakości – wyzwania Continuous ...
[TestWarez 2017] Od testowania do monitoringu jakości – wyzwania Continuous ...
 
Pitch Deck Teardown: Lumigo's $29 million Series A deck
Pitch Deck Teardown: Lumigo's $29 million Series A deckPitch Deck Teardown: Lumigo's $29 million Series A deck
Pitch Deck Teardown: Lumigo's $29 million Series A deck
 
ITESOFT Insider View | The Importance of Capture
ITESOFT Insider View | The Importance of CaptureITESOFT Insider View | The Importance of Capture
ITESOFT Insider View | The Importance of Capture
 
2016 Yahoo Taiwan Mobile Developer Conference
2016 Yahoo Taiwan Mobile Developer Conference 2016 Yahoo Taiwan Mobile Developer Conference
2016 Yahoo Taiwan Mobile Developer Conference
 

More from CrowdFlower

Building Better Models Faster Using Active Learning
Building Better Models Faster Using Active LearningBuilding Better Models Faster Using Active Learning
Building Better Models Faster Using Active LearningCrowdFlower
 
CrowdFlower NDA Crowds - Secure, exceptional tasking at a massive scale.
CrowdFlower NDA Crowds - Secure, exceptional tasking at a massive scale. CrowdFlower NDA Crowds - Secure, exceptional tasking at a massive scale.
CrowdFlower NDA Crowds - Secure, exceptional tasking at a massive scale. CrowdFlower
 
CrowdFlower Product Webinar - Graphical Editor and Visual Reports
CrowdFlower Product Webinar - Graphical Editor and Visual ReportsCrowdFlower Product Webinar - Graphical Editor and Visual Reports
CrowdFlower Product Webinar - Graphical Editor and Visual ReportsCrowdFlower
 
How Oracle Uses CrowdFlower For Sentiment Analysis
How Oracle Uses CrowdFlower For Sentiment AnalysisHow Oracle Uses CrowdFlower For Sentiment Analysis
How Oracle Uses CrowdFlower For Sentiment AnalysisCrowdFlower
 
Humanizing The Machine
Humanizing The MachineHumanizing The Machine
Humanizing The MachineCrowdFlower
 
Productive Out-of-the-Box | Tooling with Yeoman to Rapidly Develop Ember.js A...
Productive Out-of-the-Box | Tooling with Yeoman to Rapidly Develop Ember.js A...Productive Out-of-the-Box | Tooling with Yeoman to Rapidly Develop Ember.js A...
Productive Out-of-the-Box | Tooling with Yeoman to Rapidly Develop Ember.js A...CrowdFlower
 
Virtual Data Steward: Data Management 3.0
Virtual Data Steward: Data Management 3.0Virtual Data Steward: Data Management 3.0
Virtual Data Steward: Data Management 3.0CrowdFlower
 
Expert Crowdsourcing with Flash Teams | CrowdConf 2013 poster
Expert Crowdsourcing with Flash Teams | CrowdConf 2013 posterExpert Crowdsourcing with Flash Teams | CrowdConf 2013 poster
Expert Crowdsourcing with Flash Teams | CrowdConf 2013 posterCrowdFlower
 
The State of Enterprise Crowdsourcing 2013
The State of Enterprise Crowdsourcing 2013The State of Enterprise Crowdsourcing 2013
The State of Enterprise Crowdsourcing 2013CrowdFlower
 
CrowdFlower University Oct. 21 2013
CrowdFlower University Oct. 21 2013CrowdFlower University Oct. 21 2013
CrowdFlower University Oct. 21 2013CrowdFlower
 

More from CrowdFlower (11)

7 Myths of AI
7 Myths of AI7 Myths of AI
7 Myths of AI
 
Building Better Models Faster Using Active Learning
Building Better Models Faster Using Active LearningBuilding Better Models Faster Using Active Learning
Building Better Models Faster Using Active Learning
 
CrowdFlower NDA Crowds - Secure, exceptional tasking at a massive scale.
CrowdFlower NDA Crowds - Secure, exceptional tasking at a massive scale. CrowdFlower NDA Crowds - Secure, exceptional tasking at a massive scale.
CrowdFlower NDA Crowds - Secure, exceptional tasking at a massive scale.
 
CrowdFlower Product Webinar - Graphical Editor and Visual Reports
CrowdFlower Product Webinar - Graphical Editor and Visual ReportsCrowdFlower Product Webinar - Graphical Editor and Visual Reports
CrowdFlower Product Webinar - Graphical Editor and Visual Reports
 
How Oracle Uses CrowdFlower For Sentiment Analysis
How Oracle Uses CrowdFlower For Sentiment AnalysisHow Oracle Uses CrowdFlower For Sentiment Analysis
How Oracle Uses CrowdFlower For Sentiment Analysis
 
Humanizing The Machine
Humanizing The MachineHumanizing The Machine
Humanizing The Machine
 
Productive Out-of-the-Box | Tooling with Yeoman to Rapidly Develop Ember.js A...
Productive Out-of-the-Box | Tooling with Yeoman to Rapidly Develop Ember.js A...Productive Out-of-the-Box | Tooling with Yeoman to Rapidly Develop Ember.js A...
Productive Out-of-the-Box | Tooling with Yeoman to Rapidly Develop Ember.js A...
 
Virtual Data Steward: Data Management 3.0
Virtual Data Steward: Data Management 3.0Virtual Data Steward: Data Management 3.0
Virtual Data Steward: Data Management 3.0
 
Expert Crowdsourcing with Flash Teams | CrowdConf 2013 poster
Expert Crowdsourcing with Flash Teams | CrowdConf 2013 posterExpert Crowdsourcing with Flash Teams | CrowdConf 2013 poster
Expert Crowdsourcing with Flash Teams | CrowdConf 2013 poster
 
The State of Enterprise Crowdsourcing 2013
The State of Enterprise Crowdsourcing 2013The State of Enterprise Crowdsourcing 2013
The State of Enterprise Crowdsourcing 2013
 
CrowdFlower University Oct. 21 2013
CrowdFlower University Oct. 21 2013CrowdFlower University Oct. 21 2013
CrowdFlower University Oct. 21 2013
 

Recently uploaded

modul pembelajaran robotic Workshop _ by Slidesgo.pptx
modul pembelajaran robotic Workshop _ by Slidesgo.pptxmodul pembelajaran robotic Workshop _ by Slidesgo.pptx
modul pembelajaran robotic Workshop _ by Slidesgo.pptxaleedritatuxx
 
Predicting Salary Using Data Science: A Comprehensive Analysis.pdf
Predicting Salary Using Data Science: A Comprehensive Analysis.pdfPredicting Salary Using Data Science: A Comprehensive Analysis.pdf
Predicting Salary Using Data Science: A Comprehensive Analysis.pdfBoston Institute of Analytics
 
RS 9000 Call In girls Dwarka Mor (DELHI)⇛9711147426🔝Delhi
RS 9000 Call In girls Dwarka Mor (DELHI)⇛9711147426🔝DelhiRS 9000 Call In girls Dwarka Mor (DELHI)⇛9711147426🔝Delhi
RS 9000 Call In girls Dwarka Mor (DELHI)⇛9711147426🔝Delhijennyeacort
 
办理(UC毕业证书)堪培拉大学毕业证成绩单原版一比一
办理(UC毕业证书)堪培拉大学毕业证成绩单原版一比一办理(UC毕业证书)堪培拉大学毕业证成绩单原版一比一
办理(UC毕业证书)堪培拉大学毕业证成绩单原版一比一z xss
 
Digital Marketing Plan, how digital marketing works
Digital Marketing Plan, how digital marketing worksDigital Marketing Plan, how digital marketing works
Digital Marketing Plan, how digital marketing worksdeepakthakur548787
 
办美国阿肯色大学小石城分校毕业证成绩单pdf电子版制作修改#真实留信入库#永久存档#真实可查#diploma#degree
办美国阿肯色大学小石城分校毕业证成绩单pdf电子版制作修改#真实留信入库#永久存档#真实可查#diploma#degree办美国阿肯色大学小石城分校毕业证成绩单pdf电子版制作修改#真实留信入库#永久存档#真实可查#diploma#degree
办美国阿肯色大学小石城分校毕业证成绩单pdf电子版制作修改#真实留信入库#永久存档#真实可查#diploma#degreeyuu sss
 
ASML's Taxonomy Adventure by Daniel Canter
ASML's Taxonomy Adventure by Daniel CanterASML's Taxonomy Adventure by Daniel Canter
ASML's Taxonomy Adventure by Daniel Cantervoginip
 
Data Analysis Project : Targeting the Right Customers, Presentation on Bank M...
Data Analysis Project : Targeting the Right Customers, Presentation on Bank M...Data Analysis Project : Targeting the Right Customers, Presentation on Bank M...
Data Analysis Project : Targeting the Right Customers, Presentation on Bank M...Boston Institute of Analytics
 
FAIR, FAIRsharing, FAIR Cookbook and ELIXIR - Sansone SA - Boston 2024
FAIR, FAIRsharing, FAIR Cookbook and ELIXIR - Sansone SA - Boston 2024FAIR, FAIRsharing, FAIR Cookbook and ELIXIR - Sansone SA - Boston 2024
FAIR, FAIRsharing, FAIR Cookbook and ELIXIR - Sansone SA - Boston 2024Susanna-Assunta Sansone
 
Bank Loan Approval Analysis: A Comprehensive Data Analysis Project
Bank Loan Approval Analysis: A Comprehensive Data Analysis ProjectBank Loan Approval Analysis: A Comprehensive Data Analysis Project
Bank Loan Approval Analysis: A Comprehensive Data Analysis ProjectBoston Institute of Analytics
 
INTRODUCTION TO Natural language processing
INTRODUCTION TO Natural language processingINTRODUCTION TO Natural language processing
INTRODUCTION TO Natural language processingsocarem879
 
Learn How Data Science Changes Our World
Learn How Data Science Changes Our WorldLearn How Data Science Changes Our World
Learn How Data Science Changes Our WorldEduminds Learning
 
Minimizing AI Hallucinations/Confabulations and the Path towards AGI with Exa...
Minimizing AI Hallucinations/Confabulations and the Path towards AGI with Exa...Minimizing AI Hallucinations/Confabulations and the Path towards AGI with Exa...
Minimizing AI Hallucinations/Confabulations and the Path towards AGI with Exa...Thomas Poetter
 
Top 5 Best Data Analytics Courses In Queens
Top 5 Best Data Analytics Courses In QueensTop 5 Best Data Analytics Courses In Queens
Top 5 Best Data Analytics Courses In Queensdataanalyticsqueen03
 
毕业文凭制作#回国入职#diploma#degree澳洲中央昆士兰大学毕业证成绩单pdf电子版制作修改#毕业文凭制作#回国入职#diploma#degree
毕业文凭制作#回国入职#diploma#degree澳洲中央昆士兰大学毕业证成绩单pdf电子版制作修改#毕业文凭制作#回国入职#diploma#degree毕业文凭制作#回国入职#diploma#degree澳洲中央昆士兰大学毕业证成绩单pdf电子版制作修改#毕业文凭制作#回国入职#diploma#degree
毕业文凭制作#回国入职#diploma#degree澳洲中央昆士兰大学毕业证成绩单pdf电子版制作修改#毕业文凭制作#回国入职#diploma#degreeyuu sss
 
English-8-Q4-W3-Synthesizing-Essential-Information-From-Various-Sources-1.pdf
English-8-Q4-W3-Synthesizing-Essential-Information-From-Various-Sources-1.pdfEnglish-8-Q4-W3-Synthesizing-Essential-Information-From-Various-Sources-1.pdf
English-8-Q4-W3-Synthesizing-Essential-Information-From-Various-Sources-1.pdfblazblazml
 
Conf42-LLM_Adding Generative AI to Real-Time Streaming Pipelines
Conf42-LLM_Adding Generative AI to Real-Time Streaming PipelinesConf42-LLM_Adding Generative AI to Real-Time Streaming Pipelines
Conf42-LLM_Adding Generative AI to Real-Time Streaming PipelinesTimothy Spann
 
Identifying Appropriate Test Statistics Involving Population Mean
Identifying Appropriate Test Statistics Involving Population MeanIdentifying Appropriate Test Statistics Involving Population Mean
Identifying Appropriate Test Statistics Involving Population MeanMYRABACSAFRA2
 
Decoding Patterns: Customer Churn Prediction Data Analysis Project
Decoding Patterns: Customer Churn Prediction Data Analysis ProjectDecoding Patterns: Customer Churn Prediction Data Analysis Project
Decoding Patterns: Customer Churn Prediction Data Analysis ProjectBoston Institute of Analytics
 

Recently uploaded (20)

modul pembelajaran robotic Workshop _ by Slidesgo.pptx
modul pembelajaran robotic Workshop _ by Slidesgo.pptxmodul pembelajaran robotic Workshop _ by Slidesgo.pptx
modul pembelajaran robotic Workshop _ by Slidesgo.pptx
 
Predicting Salary Using Data Science: A Comprehensive Analysis.pdf
Predicting Salary Using Data Science: A Comprehensive Analysis.pdfPredicting Salary Using Data Science: A Comprehensive Analysis.pdf
Predicting Salary Using Data Science: A Comprehensive Analysis.pdf
 
RS 9000 Call In girls Dwarka Mor (DELHI)⇛9711147426🔝Delhi
RS 9000 Call In girls Dwarka Mor (DELHI)⇛9711147426🔝DelhiRS 9000 Call In girls Dwarka Mor (DELHI)⇛9711147426🔝Delhi
RS 9000 Call In girls Dwarka Mor (DELHI)⇛9711147426🔝Delhi
 
办理(UC毕业证书)堪培拉大学毕业证成绩单原版一比一
办理(UC毕业证书)堪培拉大学毕业证成绩单原版一比一办理(UC毕业证书)堪培拉大学毕业证成绩单原版一比一
办理(UC毕业证书)堪培拉大学毕业证成绩单原版一比一
 
Digital Marketing Plan, how digital marketing works
Digital Marketing Plan, how digital marketing worksDigital Marketing Plan, how digital marketing works
Digital Marketing Plan, how digital marketing works
 
办美国阿肯色大学小石城分校毕业证成绩单pdf电子版制作修改#真实留信入库#永久存档#真实可查#diploma#degree
办美国阿肯色大学小石城分校毕业证成绩单pdf电子版制作修改#真实留信入库#永久存档#真实可查#diploma#degree办美国阿肯色大学小石城分校毕业证成绩单pdf电子版制作修改#真实留信入库#永久存档#真实可查#diploma#degree
办美国阿肯色大学小石城分校毕业证成绩单pdf电子版制作修改#真实留信入库#永久存档#真实可查#diploma#degree
 
ASML's Taxonomy Adventure by Daniel Canter
ASML's Taxonomy Adventure by Daniel CanterASML's Taxonomy Adventure by Daniel Canter
ASML's Taxonomy Adventure by Daniel Canter
 
Data Analysis Project : Targeting the Right Customers, Presentation on Bank M...
Data Analysis Project : Targeting the Right Customers, Presentation on Bank M...Data Analysis Project : Targeting the Right Customers, Presentation on Bank M...
Data Analysis Project : Targeting the Right Customers, Presentation on Bank M...
 
FAIR, FAIRsharing, FAIR Cookbook and ELIXIR - Sansone SA - Boston 2024
FAIR, FAIRsharing, FAIR Cookbook and ELIXIR - Sansone SA - Boston 2024FAIR, FAIRsharing, FAIR Cookbook and ELIXIR - Sansone SA - Boston 2024
FAIR, FAIRsharing, FAIR Cookbook and ELIXIR - Sansone SA - Boston 2024
 
Bank Loan Approval Analysis: A Comprehensive Data Analysis Project
Bank Loan Approval Analysis: A Comprehensive Data Analysis ProjectBank Loan Approval Analysis: A Comprehensive Data Analysis Project
Bank Loan Approval Analysis: A Comprehensive Data Analysis Project
 
INTRODUCTION TO Natural language processing
INTRODUCTION TO Natural language processingINTRODUCTION TO Natural language processing
INTRODUCTION TO Natural language processing
 
Learn How Data Science Changes Our World
Learn How Data Science Changes Our WorldLearn How Data Science Changes Our World
Learn How Data Science Changes Our World
 
Insurance Churn Prediction Data Analysis Project
Insurance Churn Prediction Data Analysis ProjectInsurance Churn Prediction Data Analysis Project
Insurance Churn Prediction Data Analysis Project
 
Minimizing AI Hallucinations/Confabulations and the Path towards AGI with Exa...
Minimizing AI Hallucinations/Confabulations and the Path towards AGI with Exa...Minimizing AI Hallucinations/Confabulations and the Path towards AGI with Exa...
Minimizing AI Hallucinations/Confabulations and the Path towards AGI with Exa...
 
Top 5 Best Data Analytics Courses In Queens
Top 5 Best Data Analytics Courses In QueensTop 5 Best Data Analytics Courses In Queens
Top 5 Best Data Analytics Courses In Queens
 
毕业文凭制作#回国入职#diploma#degree澳洲中央昆士兰大学毕业证成绩单pdf电子版制作修改#毕业文凭制作#回国入职#diploma#degree
毕业文凭制作#回国入职#diploma#degree澳洲中央昆士兰大学毕业证成绩单pdf电子版制作修改#毕业文凭制作#回国入职#diploma#degree毕业文凭制作#回国入职#diploma#degree澳洲中央昆士兰大学毕业证成绩单pdf电子版制作修改#毕业文凭制作#回国入职#diploma#degree
毕业文凭制作#回国入职#diploma#degree澳洲中央昆士兰大学毕业证成绩单pdf电子版制作修改#毕业文凭制作#回国入职#diploma#degree
 
English-8-Q4-W3-Synthesizing-Essential-Information-From-Various-Sources-1.pdf
English-8-Q4-W3-Synthesizing-Essential-Information-From-Various-Sources-1.pdfEnglish-8-Q4-W3-Synthesizing-Essential-Information-From-Various-Sources-1.pdf
English-8-Q4-W3-Synthesizing-Essential-Information-From-Various-Sources-1.pdf
 
Conf42-LLM_Adding Generative AI to Real-Time Streaming Pipelines
Conf42-LLM_Adding Generative AI to Real-Time Streaming PipelinesConf42-LLM_Adding Generative AI to Real-Time Streaming Pipelines
Conf42-LLM_Adding Generative AI to Real-Time Streaming Pipelines
 
Identifying Appropriate Test Statistics Involving Population Mean
Identifying Appropriate Test Statistics Involving Population MeanIdentifying Appropriate Test Statistics Involving Population Mean
Identifying Appropriate Test Statistics Involving Population Mean
 
Decoding Patterns: Customer Churn Prediction Data Analysis Project
Decoding Patterns: Customer Churn Prediction Data Analysis ProjectDecoding Patterns: Customer Churn Prediction Data Analysis Project
Decoding Patterns: Customer Churn Prediction Data Analysis Project
 

Active Learning and Human-in-the-Loop

Editor's Notes

  1. Over 200,000 Records
  2. 59,000 records
  3. Unlike humans, artificial intelligence has no ego, so it can make an unbiased estimate of its confidence - Where it’s confident we use its answer, because hardware CPUs get cheaper+faster every year and human CPUs don’t - Where it’s not confident we use a human because in real business applications 80% accuracy isn’t good enough
  4. I think we can and should apply this to every business process We start with a machine learning classifier. Unlike humans, artificial intelligence has no ego, so it can make an unbiased estimate of its confidence - Where it’s confident we use its answer, because hardware CPUs get cheaper+faster every year and human CPUs don’t - Where it’s not confident we use a human because in real business applications 80% accuracy isn’t good enough A huge side benefit is that the human labels can be reused used to improve the machine learning classifier over time. We didn’t invent any of this, lot’s of people are talking about this and thinking about this, including many people in the room. But looking at the industry we see a lot more people talking about it than actually doing it. We are going to make this setup so easy that you will have no excuse for not doing it.
  5. I think we can and should apply this to every business process We start with a machine learning classifier. Unlike humans, artificial intelligence has no ego, so it can make an unbiased estimate of its confidence - Where it’s confident we use its answer, because hardware CPUs get cheaper+faster every year and human CPUs don’t - Where it’s not confident we use a human because in real business applications 80% accuracy isn’t good enough A huge side benefit is that the human labels can be reused used to improve the machine learning classifier over time. We didn’t invent any of this, lot’s of people are talking about this and thinking about this, including many people in the room. But looking at the industry we see a lot more people talking about it than actually doing it. We are going to make this setup so easy that you will have no excuse for not doing it.
  6. A huge side benefit is that the human labels can be reused used to improve the machine learning classifier over time.
  7. handed control to the driver 272 times and a test driver felt compelled to intervene 69 times
  8. In the field of chess computers passed humans a long time ago. But if you really want to make a great chess playing algorithm you would still use a human and computer together. There is a subculture of folks who still play “Advanced Chess” and this is actually where the highest quality chess games take place. - Still situations where humans are better