SlideShare a Scribd company logo
@vinaya1980 @hrishikarekar
TESTING IN THE AGE OF MACHINE LEARNING
March 2016
https://www.youtube.com/watch?v=DCgHsxISE0Q
W H E R E A R E
W E H E A D E D
httpsweforum.org/agenda/2017/01/worried-
about-ai-taking-your-job-its-already-happening-in-
japan?utm_content=buffera14af&utm_medium=s
ocial&utm_source=twitter.com&utm_campaign=b
uffer
THE WORLD OF ARTIFICIAL INTELLIGENCE
NLP
Neural Networks
MACHINE LEARNING
DEEP LEARNING
…..
THE FUTURE IS ALREADY HERE
Google RankBrain
Assistants
HOW DOES IT WORK
Supervised
• Labelled data
• Given new
data, predict
outcome
• Classification
Unsupervised
• No labels
• Find hidden
structures
• Clustering
Reinforcement
• Decision
process
• Actions are
rewarded or
punished
• Learns to
optimize
rewards
http://scikit-learn.org/
LEARNING FROM DATA
SUPERVISED LEARNING - CLASSIFCATION
http://www.mikedeff.in/MLIntro.PNG
INTO THE REALM OF PROBABILITIES
Y = f ( x ) Y ≈ f ( x )
What is scrum ?
{
"Prediction": {
"details": {
"Algorithm": "SGD",
"PredictiveModelType": "MULTICLASS"
},
"predictedLabel": "definition",
"predictedScores": {
"advantages": 0.0001860455668065697,
"characteristics": 0.00006915141420904547,
"compare": 0.00017757616296876222,
"definition": 0.9970965385437012,
"disadvantages": 0.0000534967657586094,
}
}
}
Can you tell me about scrum ?
{
"Prediction": {
"details": {
"Algorithm": "SGD",
"PredictiveModelType": "MULTICLASS"
},
"predictedLabel": "definition",
"predictedScores": {
"advantages": 0.01977257989346981,
"characteristics": 0.022757112979888916,
"compare": 0.008386141620576382,
"definition": 0.21092116832733154,
"disadvantages": 0.04002799838781357
}
}
}
TOLERANCE LEVELS
Y ≈ f ( x )
Know the probability that is within acceptable limits
EVALUATE WITH DIFFERENT MODELS
Evaluate against a set of
algorithms to iterate towards a
model that’s closest
representation and for further
tuning
https://s3.amazonaws.com/MLMastery/MachineLearningAlgorithms.png?__s=h4reg8jqwyg4sz3bzdqf
EVALUATION – DATA SET APPROACHES
Random split
• 70% train, 30% test
K-fold cross validation
Split into 3 datasets
• #1 Train on 1 and 2, test on 3
• #2 Train on 2 and 3, test on 1
• #3 Train on 1 and 3, test on 2
Never use the same dataset for training and evaluating
ITERATIVE – LEARNING PROCESS
Be prepared to throw the model and start again 
MODEL ACCURACY - CONFUSION MATRIX
Strive for better models, not 100% accurate.
OVERFITTING
MODEL IS AS GOOD AS THE TRAINING DATA
If all of the algorithms perform poorly,
• it maybe worth considering if there is a lack of learning
structure in the data set
• some transformation needed to make the structure more
learnable
• remove unnecessary noise - stop words are typically
removed because they cause unnecessary noise)
SUMMARY
Machine Learning applications demand a shift in testing
approach
• Use objective acceptance levels to evaluate the application
• Express test outcomes in statistical terms
• Have a high level understanding of the underlying working of
the application
@vinaya1980 @hrishikarekar
THANK YOU

More Related Content

What's hot

What is Regression Testing? | Edureka
What is Regression Testing? | EdurekaWhat is Regression Testing? | Edureka
What is Regression Testing? | Edureka
Edureka!
 
Testing Metrics - Making your tests visible
Testing Metrics - Making your tests visibleTesting Metrics - Making your tests visible
Testing Metrics - Making your tests visible
Alper Mermer
 
4.4.2013 Software Quality - Regression Testing Automated and Manual - RFT/RQM
4.4.2013 Software Quality - Regression Testing Automated and Manual - RFT/RQM4.4.2013 Software Quality - Regression Testing Automated and Manual - RFT/RQM
4.4.2013 Software Quality - Regression Testing Automated and Manual - RFT/RQM
IBM Rational
 
How QA engineers could affect quality?
How QA engineers could affect quality?How QA engineers could affect quality?
How QA engineers could affect quality?
Mikalai Alimenkou
 
Pitfalls of Test Automation
Pitfalls of Test AutomationPitfalls of Test Automation
Pitfalls of Test Automation
Reload! A/S
 
Top 5 pitfalls of software test automatiion
Top 5 pitfalls of software test automatiionTop 5 pitfalls of software test automatiion
Top 5 pitfalls of software test automatiion
ekatechserv
 
Automated visual-regression-testing (1)
Automated visual-regression-testing (1)Automated visual-regression-testing (1)
Automated visual-regression-testing (1)
Sriram Angajala
 
End-to-End Test Automation for Both Horizontal and Vertical Scale
End-to-End Test Automation for Both Horizontal and Vertical ScaleEnd-to-End Test Automation for Both Horizontal and Vertical Scale
End-to-End Test Automation for Both Horizontal and Vertical Scale
Erdem YILDIRIM
 
Understand regression testing
Understand regression testingUnderstand regression testing
Understand regression testing
gaoliang641
 
It Seemed a Good Idea at the Time: Intelligent Mistakes in Test Automation
It Seemed a Good Idea at the Time: Intelligent Mistakes in Test AutomationIt Seemed a Good Idea at the Time: Intelligent Mistakes in Test Automation
It Seemed a Good Idea at the Time: Intelligent Mistakes in Test Automation
TechWell
 
Solano Labs presented at MassTLC's automated testing
Solano Labs presented at MassTLC's automated testingSolano Labs presented at MassTLC's automated testing
Solano Labs presented at MassTLC's automated testing
MassTLC
 
Defect root cause analysis, Андрей Титаренко
Defect root cause analysis, Андрей ТитаренкоDefect root cause analysis, Андрей Титаренко
Defect root cause analysis, Андрей Титаренко
Sigma Software
 
Wso2con test-automation
Wso2con test-automationWso2con test-automation
Wso2con test-automation
krishantha_samaraweera
 
End to end testing - strategies
End to end testing - strategiesEnd to end testing - strategies
End to end testing - strategies
anuvip
 
Presentation1
Presentation1Presentation1
Presentation1
anuvip
 
Management Issues in Test Automation
Management Issues in Test AutomationManagement Issues in Test Automation
Management Issues in Test Automation
TechWell
 
Regression testing
Regression testingRegression testing
Regression testing
Anamta Sayyed
 
ISTQB Advanced Test Manager Training
ISTQB Advanced Test Manager TrainingISTQB Advanced Test Manager Training
ISTQB Advanced Test Manager Training
HiraQureshi22
 
How Automation is Changing The Testing Scene
How Automation is Changing The Testing SceneHow Automation is Changing The Testing Scene
How Automation is Changing The Testing Scene
suneratechnologies
 
Crawl Walk and Run to Continuous Delivery
Crawl Walk and Run to Continuous DeliveryCrawl Walk and Run to Continuous Delivery
Crawl Walk and Run to Continuous Delivery
David Batten
 

What's hot (20)

What is Regression Testing? | Edureka
What is Regression Testing? | EdurekaWhat is Regression Testing? | Edureka
What is Regression Testing? | Edureka
 
Testing Metrics - Making your tests visible
Testing Metrics - Making your tests visibleTesting Metrics - Making your tests visible
Testing Metrics - Making your tests visible
 
4.4.2013 Software Quality - Regression Testing Automated and Manual - RFT/RQM
4.4.2013 Software Quality - Regression Testing Automated and Manual - RFT/RQM4.4.2013 Software Quality - Regression Testing Automated and Manual - RFT/RQM
4.4.2013 Software Quality - Regression Testing Automated and Manual - RFT/RQM
 
How QA engineers could affect quality?
How QA engineers could affect quality?How QA engineers could affect quality?
How QA engineers could affect quality?
 
Pitfalls of Test Automation
Pitfalls of Test AutomationPitfalls of Test Automation
Pitfalls of Test Automation
 
Top 5 pitfalls of software test automatiion
Top 5 pitfalls of software test automatiionTop 5 pitfalls of software test automatiion
Top 5 pitfalls of software test automatiion
 
Automated visual-regression-testing (1)
Automated visual-regression-testing (1)Automated visual-regression-testing (1)
Automated visual-regression-testing (1)
 
End-to-End Test Automation for Both Horizontal and Vertical Scale
End-to-End Test Automation for Both Horizontal and Vertical ScaleEnd-to-End Test Automation for Both Horizontal and Vertical Scale
End-to-End Test Automation for Both Horizontal and Vertical Scale
 
Understand regression testing
Understand regression testingUnderstand regression testing
Understand regression testing
 
It Seemed a Good Idea at the Time: Intelligent Mistakes in Test Automation
It Seemed a Good Idea at the Time: Intelligent Mistakes in Test AutomationIt Seemed a Good Idea at the Time: Intelligent Mistakes in Test Automation
It Seemed a Good Idea at the Time: Intelligent Mistakes in Test Automation
 
Solano Labs presented at MassTLC's automated testing
Solano Labs presented at MassTLC's automated testingSolano Labs presented at MassTLC's automated testing
Solano Labs presented at MassTLC's automated testing
 
Defect root cause analysis, Андрей Титаренко
Defect root cause analysis, Андрей ТитаренкоDefect root cause analysis, Андрей Титаренко
Defect root cause analysis, Андрей Титаренко
 
Wso2con test-automation
Wso2con test-automationWso2con test-automation
Wso2con test-automation
 
End to end testing - strategies
End to end testing - strategiesEnd to end testing - strategies
End to end testing - strategies
 
Presentation1
Presentation1Presentation1
Presentation1
 
Management Issues in Test Automation
Management Issues in Test AutomationManagement Issues in Test Automation
Management Issues in Test Automation
 
Regression testing
Regression testingRegression testing
Regression testing
 
ISTQB Advanced Test Manager Training
ISTQB Advanced Test Manager TrainingISTQB Advanced Test Manager Training
ISTQB Advanced Test Manager Training
 
How Automation is Changing The Testing Scene
How Automation is Changing The Testing SceneHow Automation is Changing The Testing Scene
How Automation is Changing The Testing Scene
 
Crawl Walk and Run to Continuous Delivery
Crawl Walk and Run to Continuous DeliveryCrawl Walk and Run to Continuous Delivery
Crawl Walk and Run to Continuous Delivery
 

Viewers also liked

The Snowball Effect
The Snowball EffectThe Snowball Effect
The Snowball Effect
vinaya muralidharan
 
B2 - Integrating on-premises workloads with AWS
B2 - Integrating on-premises workloads with AWSB2 - Integrating on-premises workloads with AWS
B2 - Integrating on-premises workloads with AWS
Amazon Web Services
 
Orchestration & Deployment Options for Hybrid Enterprise Environments (ARC310...
Orchestration & Deployment Options for Hybrid Enterprise Environments (ARC310...Orchestration & Deployment Options for Hybrid Enterprise Environments (ARC310...
Orchestration & Deployment Options for Hybrid Enterprise Environments (ARC310...
Amazon Web Services
 
Testing machine learning, learning machine testing. EuroSTAR 2016 Rik Marselis
Testing machine learning, learning machine testing. EuroSTAR 2016 Rik MarselisTesting machine learning, learning machine testing. EuroSTAR 2016 Rik Marselis
Testing machine learning, learning machine testing. EuroSTAR 2016 Rik Marselis
Rik Marselis
 
AWS and VMware: How to Architect and Manage Hybrid Environments
AWS and VMware: How to Architect and Manage Hybrid EnvironmentsAWS and VMware: How to Architect and Manage Hybrid Environments
AWS and VMware: How to Architect and Manage Hybrid Environments
RightScale
 
AWS Summit Stockholm 2014 – B3 – Integrating on-premises workloads with AWS
AWS Summit Stockholm 2014 – B3 – Integrating on-premises workloads with AWSAWS Summit Stockholm 2014 – B3 – Integrating on-premises workloads with AWS
AWS Summit Stockholm 2014 – B3 – Integrating on-premises workloads with AWS
Amazon Web Services
 
ATAGTR2017 Machine Learning telepathy for Shift Right approach of testing
ATAGTR2017 Machine Learning telepathy for Shift Right approach of testingATAGTR2017 Machine Learning telepathy for Shift Right approach of testing
ATAGTR2017 Machine Learning telepathy for Shift Right approach of testing
Agile Testing Alliance
 
Disruptive technologies that will change aviation in the coming years
Disruptive technologies that will change aviation in the coming yearsDisruptive technologies that will change aviation in the coming years
Disruptive technologies that will change aviation in the coming years
Thoughtworks
 
Machine Learning on Big Data
Machine Learning on Big DataMachine Learning on Big Data
Machine Learning on Big Data
Max Lin
 
AI for IA's: Machine Learning Demystified at IA Summit 2017 - IAS17
AI for IA's: Machine Learning Demystified at IA Summit 2017 - IAS17AI for IA's: Machine Learning Demystified at IA Summit 2017 - IAS17
AI for IA's: Machine Learning Demystified at IA Summit 2017 - IAS17
Carol Smith
 

Viewers also liked (10)

The Snowball Effect
The Snowball EffectThe Snowball Effect
The Snowball Effect
 
B2 - Integrating on-premises workloads with AWS
B2 - Integrating on-premises workloads with AWSB2 - Integrating on-premises workloads with AWS
B2 - Integrating on-premises workloads with AWS
 
Orchestration & Deployment Options for Hybrid Enterprise Environments (ARC310...
Orchestration & Deployment Options for Hybrid Enterprise Environments (ARC310...Orchestration & Deployment Options for Hybrid Enterprise Environments (ARC310...
Orchestration & Deployment Options for Hybrid Enterprise Environments (ARC310...
 
Testing machine learning, learning machine testing. EuroSTAR 2016 Rik Marselis
Testing machine learning, learning machine testing. EuroSTAR 2016 Rik MarselisTesting machine learning, learning machine testing. EuroSTAR 2016 Rik Marselis
Testing machine learning, learning machine testing. EuroSTAR 2016 Rik Marselis
 
AWS and VMware: How to Architect and Manage Hybrid Environments
AWS and VMware: How to Architect and Manage Hybrid EnvironmentsAWS and VMware: How to Architect and Manage Hybrid Environments
AWS and VMware: How to Architect and Manage Hybrid Environments
 
AWS Summit Stockholm 2014 – B3 – Integrating on-premises workloads with AWS
AWS Summit Stockholm 2014 – B3 – Integrating on-premises workloads with AWSAWS Summit Stockholm 2014 – B3 – Integrating on-premises workloads with AWS
AWS Summit Stockholm 2014 – B3 – Integrating on-premises workloads with AWS
 
ATAGTR2017 Machine Learning telepathy for Shift Right approach of testing
ATAGTR2017 Machine Learning telepathy for Shift Right approach of testingATAGTR2017 Machine Learning telepathy for Shift Right approach of testing
ATAGTR2017 Machine Learning telepathy for Shift Right approach of testing
 
Disruptive technologies that will change aviation in the coming years
Disruptive technologies that will change aviation in the coming yearsDisruptive technologies that will change aviation in the coming years
Disruptive technologies that will change aviation in the coming years
 
Machine Learning on Big Data
Machine Learning on Big DataMachine Learning on Big Data
Machine Learning on Big Data
 
AI for IA's: Machine Learning Demystified at IA Summit 2017 - IAS17
AI for IA's: Machine Learning Demystified at IA Summit 2017 - IAS17AI for IA's: Machine Learning Demystified at IA Summit 2017 - IAS17
AI for IA's: Machine Learning Demystified at IA Summit 2017 - IAS17
 

Similar to Testing in the Age of Machine Learning

1030 track1 bennett
1030 track1 bennett1030 track1 bennett
1030 track1 bennett
Rising Media, Inc.
 
supervised and unsupervised learning
supervised and unsupervised learningsupervised and unsupervised learning
supervised and unsupervised learning
shivani saluja
 
Presentation slides 2
Presentation slides 2Presentation slides 2
Presentation slides 2
HanimHanem
 
AzureML – zero to hero
AzureML – zero to heroAzureML – zero to hero
AzureML – zero to hero
Govind Kanshi
 
The path to be a data scientist
The path to be a data scientistThe path to be a data scientist
The path to be a data scientist
Poo Kuan Hoong
 
Introduction to Machine Learning
Introduction to Machine LearningIntroduction to Machine Learning
Introduction to Machine Learning
Knowledge And Skill Forum
 
Qualitative and quantitative analysis
Qualitative and quantitative analysisQualitative and quantitative analysis
Qualitative and quantitative analysis
Nellie Deutsch (Ed.D)
 
Fie2014 guenaga etal
Fie2014 guenaga etalFie2014 guenaga etal
Fie2014 guenaga etal
Mª Luz Guenaga Gómez
 
Datascience methodology
Datascience methodologyDatascience methodology
Datascience methodology
ArunakumariAkula1
 
LETS PUBLISH WITH MORE RELIABLE & PRESENTABLE MODELLING.pptx
LETS PUBLISH WITH MORE RELIABLE & PRESENTABLE MODELLING.pptxLETS PUBLISH WITH MORE RELIABLE & PRESENTABLE MODELLING.pptx
LETS PUBLISH WITH MORE RELIABLE & PRESENTABLE MODELLING.pptx
shamsul2010
 
Machine learning project_promotion
Machine learning project_promotionMachine learning project_promotion
Machine learning project_promotion
kahhuey
 
Data Collection and Analysis Tools
Data Collection and Analysis ToolsData Collection and Analysis Tools
Data Collection and Analysis Tools
Caren Gamboa
 
2018-Sogeti-TestExpo-Intelligent_Predictive_Models.pptx
2018-Sogeti-TestExpo-Intelligent_Predictive_Models.pptx2018-Sogeti-TestExpo-Intelligent_Predictive_Models.pptx
2018-Sogeti-TestExpo-Intelligent_Predictive_Models.pptx
Minh Nguyen
 
Advancing Testing Program Maturity in your organization
Advancing Testing Program Maturity in your organizationAdvancing Testing Program Maturity in your organization
Advancing Testing Program Maturity in your organization
Ramkumar Ravichandran
 
Machine learning: A Walk Through School Exams
Machine learning: A Walk Through School ExamsMachine learning: A Walk Through School Exams
Machine learning: A Walk Through School Exams
Ramsha Ijaz
 
Learning Analytics
Learning AnalyticsLearning Analytics
Learning Analytics
Brightwave Group
 
EDUC5103 7th Adobe Connect Session Presentation (March 30, 2016)
EDUC5103 7th Adobe Connect Session Presentation (March 30, 2016)EDUC5103 7th Adobe Connect Session Presentation (March 30, 2016)
EDUC5103 7th Adobe Connect Session Presentation (March 30, 2016)
Robert Power
 
MACHINE LEARNING PRESENTATION (ARTIFICIAL INTELLIGENCE)
MACHINE LEARNING PRESENTATION (ARTIFICIAL INTELLIGENCE)MACHINE LEARNING PRESENTATION (ARTIFICIAL INTELLIGENCE)
MACHINE LEARNING PRESENTATION (ARTIFICIAL INTELLIGENCE)
MAHIRA
 
Data+Science+in+Python+-+Data+Prep+&+EDA.pdf
Data+Science+in+Python+-+Data+Prep+&+EDA.pdfData+Science+in+Python+-+Data+Prep+&+EDA.pdf
Data+Science+in+Python+-+Data+Prep+&+EDA.pdf
neelakandan2001kpm
 
How to find new ways to add value to your audits
How to find new ways to add value to your auditsHow to find new ways to add value to your audits
How to find new ways to add value to your audits
CaseWare IDEA
 

Similar to Testing in the Age of Machine Learning (20)

1030 track1 bennett
1030 track1 bennett1030 track1 bennett
1030 track1 bennett
 
supervised and unsupervised learning
supervised and unsupervised learningsupervised and unsupervised learning
supervised and unsupervised learning
 
Presentation slides 2
Presentation slides 2Presentation slides 2
Presentation slides 2
 
AzureML – zero to hero
AzureML – zero to heroAzureML – zero to hero
AzureML – zero to hero
 
The path to be a data scientist
The path to be a data scientistThe path to be a data scientist
The path to be a data scientist
 
Introduction to Machine Learning
Introduction to Machine LearningIntroduction to Machine Learning
Introduction to Machine Learning
 
Qualitative and quantitative analysis
Qualitative and quantitative analysisQualitative and quantitative analysis
Qualitative and quantitative analysis
 
Fie2014 guenaga etal
Fie2014 guenaga etalFie2014 guenaga etal
Fie2014 guenaga etal
 
Datascience methodology
Datascience methodologyDatascience methodology
Datascience methodology
 
LETS PUBLISH WITH MORE RELIABLE & PRESENTABLE MODELLING.pptx
LETS PUBLISH WITH MORE RELIABLE & PRESENTABLE MODELLING.pptxLETS PUBLISH WITH MORE RELIABLE & PRESENTABLE MODELLING.pptx
LETS PUBLISH WITH MORE RELIABLE & PRESENTABLE MODELLING.pptx
 
Machine learning project_promotion
Machine learning project_promotionMachine learning project_promotion
Machine learning project_promotion
 
Data Collection and Analysis Tools
Data Collection and Analysis ToolsData Collection and Analysis Tools
Data Collection and Analysis Tools
 
2018-Sogeti-TestExpo-Intelligent_Predictive_Models.pptx
2018-Sogeti-TestExpo-Intelligent_Predictive_Models.pptx2018-Sogeti-TestExpo-Intelligent_Predictive_Models.pptx
2018-Sogeti-TestExpo-Intelligent_Predictive_Models.pptx
 
Advancing Testing Program Maturity in your organization
Advancing Testing Program Maturity in your organizationAdvancing Testing Program Maturity in your organization
Advancing Testing Program Maturity in your organization
 
Machine learning: A Walk Through School Exams
Machine learning: A Walk Through School ExamsMachine learning: A Walk Through School Exams
Machine learning: A Walk Through School Exams
 
Learning Analytics
Learning AnalyticsLearning Analytics
Learning Analytics
 
EDUC5103 7th Adobe Connect Session Presentation (March 30, 2016)
EDUC5103 7th Adobe Connect Session Presentation (March 30, 2016)EDUC5103 7th Adobe Connect Session Presentation (March 30, 2016)
EDUC5103 7th Adobe Connect Session Presentation (March 30, 2016)
 
MACHINE LEARNING PRESENTATION (ARTIFICIAL INTELLIGENCE)
MACHINE LEARNING PRESENTATION (ARTIFICIAL INTELLIGENCE)MACHINE LEARNING PRESENTATION (ARTIFICIAL INTELLIGENCE)
MACHINE LEARNING PRESENTATION (ARTIFICIAL INTELLIGENCE)
 
Data+Science+in+Python+-+Data+Prep+&+EDA.pdf
Data+Science+in+Python+-+Data+Prep+&+EDA.pdfData+Science+in+Python+-+Data+Prep+&+EDA.pdf
Data+Science+in+Python+-+Data+Prep+&+EDA.pdf
 
How to find new ways to add value to your audits
How to find new ways to add value to your auditsHow to find new ways to add value to your audits
How to find new ways to add value to your audits
 

More from vinaya muralidharan

10 surprising reasons your agile transformation is stuck
10 surprising reasons your agile transformation is stuck10 surprising reasons your agile transformation is stuck
10 surprising reasons your agile transformation is stuck
vinaya muralidharan
 
Does your product backlog include these
Does your product backlog include theseDoes your product backlog include these
Does your product backlog include these
vinaya muralidharan
 
Team formation considerations
Team formation considerationsTeam formation considerations
Team formation considerations
vinaya muralidharan
 
Bad things hide in the dark
Bad things hide in the darkBad things hide in the dark
Bad things hide in the dark
vinaya muralidharan
 
Scrumban recipe
Scrumban recipe Scrumban recipe
Scrumban recipe
vinaya muralidharan
 
An enterprise's journey towards agility
An enterprise's journey towards agility   An enterprise's journey towards agility
An enterprise's journey towards agility
vinaya muralidharan
 
Evolving to agile testing
Evolving to agile testingEvolving to agile testing
Evolving to agile testing
vinaya muralidharan
 

More from vinaya muralidharan (7)

10 surprising reasons your agile transformation is stuck
10 surprising reasons your agile transformation is stuck10 surprising reasons your agile transformation is stuck
10 surprising reasons your agile transformation is stuck
 
Does your product backlog include these
Does your product backlog include theseDoes your product backlog include these
Does your product backlog include these
 
Team formation considerations
Team formation considerationsTeam formation considerations
Team formation considerations
 
Bad things hide in the dark
Bad things hide in the darkBad things hide in the dark
Bad things hide in the dark
 
Scrumban recipe
Scrumban recipe Scrumban recipe
Scrumban recipe
 
An enterprise's journey towards agility
An enterprise's journey towards agility   An enterprise's journey towards agility
An enterprise's journey towards agility
 
Evolving to agile testing
Evolving to agile testingEvolving to agile testing
Evolving to agile testing
 

Recently uploaded

WWDC 2024 Keynote Review: For CocoaCoders Austin
WWDC 2024 Keynote Review: For CocoaCoders AustinWWDC 2024 Keynote Review: For CocoaCoders Austin
WWDC 2024 Keynote Review: For CocoaCoders Austin
Patrick Weigel
 
LORRAINE ANDREI_LEQUIGAN_HOW TO USE ZOOM
LORRAINE ANDREI_LEQUIGAN_HOW TO USE ZOOMLORRAINE ANDREI_LEQUIGAN_HOW TO USE ZOOM
LORRAINE ANDREI_LEQUIGAN_HOW TO USE ZOOM
lorraineandreiamcidl
 
Artificia Intellicence and XPath Extension Functions
Artificia Intellicence and XPath Extension FunctionsArtificia Intellicence and XPath Extension Functions
Artificia Intellicence and XPath Extension Functions
Octavian Nadolu
 
Webinar On-Demand: Using Flutter for Embedded
Webinar On-Demand: Using Flutter for EmbeddedWebinar On-Demand: Using Flutter for Embedded
Webinar On-Demand: Using Flutter for Embedded
ICS
 
E-Invoicing Implementation: A Step-by-Step Guide for Saudi Arabian Companies
E-Invoicing Implementation: A Step-by-Step Guide for Saudi Arabian CompaniesE-Invoicing Implementation: A Step-by-Step Guide for Saudi Arabian Companies
E-Invoicing Implementation: A Step-by-Step Guide for Saudi Arabian Companies
Quickdice ERP
 
How to write a program in any programming language
How to write a program in any programming languageHow to write a program in any programming language
How to write a program in any programming language
Rakesh Kumar R
 
Measures in SQL (SIGMOD 2024, Santiago, Chile)
Measures in SQL (SIGMOD 2024, Santiago, Chile)Measures in SQL (SIGMOD 2024, Santiago, Chile)
Measures in SQL (SIGMOD 2024, Santiago, Chile)
Julian Hyde
 
E-commerce Development Services- Hornet Dynamics
E-commerce Development Services- Hornet DynamicsE-commerce Development Services- Hornet Dynamics
E-commerce Development Services- Hornet Dynamics
Hornet Dynamics
 
ALGIT - Assembly Line for Green IT - Numbers, Data, Facts
ALGIT - Assembly Line for Green IT - Numbers, Data, FactsALGIT - Assembly Line for Green IT - Numbers, Data, Facts
ALGIT - Assembly Line for Green IT - Numbers, Data, Facts
Green Software Development
 
Introducing Crescat - Event Management Software for Venues, Festivals and Eve...
Introducing Crescat - Event Management Software for Venues, Festivals and Eve...Introducing Crescat - Event Management Software for Venues, Festivals and Eve...
Introducing Crescat - Event Management Software for Venues, Festivals and Eve...
Crescat
 
Unveiling the Advantages of Agile Software Development.pdf
Unveiling the Advantages of Agile Software Development.pdfUnveiling the Advantages of Agile Software Development.pdf
Unveiling the Advantages of Agile Software Development.pdf
brainerhub1
 
Top Benefits of Using Salesforce Healthcare CRM for Patient Management.pdf
Top Benefits of Using Salesforce Healthcare CRM for Patient Management.pdfTop Benefits of Using Salesforce Healthcare CRM for Patient Management.pdf
Top Benefits of Using Salesforce Healthcare CRM for Patient Management.pdf
VALiNTRY360
 
socradar-q1-2024-aviation-industry-report.pdf
socradar-q1-2024-aviation-industry-report.pdfsocradar-q1-2024-aviation-industry-report.pdf
socradar-q1-2024-aviation-industry-report.pdf
SOCRadar
 
Empowering Growth with Best Software Development Company in Noida - Deuglo
Empowering Growth with Best Software  Development Company in Noida - DeugloEmpowering Growth with Best Software  Development Company in Noida - Deuglo
Empowering Growth with Best Software Development Company in Noida - Deuglo
Deuglo Infosystem Pvt Ltd
 
Transform Your Communication with Cloud-Based IVR Solutions
Transform Your Communication with Cloud-Based IVR SolutionsTransform Your Communication with Cloud-Based IVR Solutions
Transform Your Communication with Cloud-Based IVR Solutions
TheSMSPoint
 
How Can Hiring A Mobile App Development Company Help Your Business Grow?
How Can Hiring A Mobile App Development Company Help Your Business Grow?How Can Hiring A Mobile App Development Company Help Your Business Grow?
How Can Hiring A Mobile App Development Company Help Your Business Grow?
ToXSL Technologies
 
Odoo ERP Vs. Traditional ERP Systems – A Comparative Analysis
Odoo ERP Vs. Traditional ERP Systems – A Comparative AnalysisOdoo ERP Vs. Traditional ERP Systems – A Comparative Analysis
Odoo ERP Vs. Traditional ERP Systems – A Comparative Analysis
Envertis Software Solutions
 
Requirement Traceability in Xen Functional Safety
Requirement Traceability in Xen Functional SafetyRequirement Traceability in Xen Functional Safety
Requirement Traceability in Xen Functional Safety
Ayan Halder
 
Using Query Store in Azure PostgreSQL to Understand Query Performance
Using Query Store in Azure PostgreSQL to Understand Query PerformanceUsing Query Store in Azure PostgreSQL to Understand Query Performance
Using Query Store in Azure PostgreSQL to Understand Query Performance
Grant Fritchey
 
Energy consumption of Database Management - Florina Jonuzi
Energy consumption of Database Management - Florina JonuziEnergy consumption of Database Management - Florina Jonuzi
Energy consumption of Database Management - Florina Jonuzi
Green Software Development
 

Recently uploaded (20)

WWDC 2024 Keynote Review: For CocoaCoders Austin
WWDC 2024 Keynote Review: For CocoaCoders AustinWWDC 2024 Keynote Review: For CocoaCoders Austin
WWDC 2024 Keynote Review: For CocoaCoders Austin
 
LORRAINE ANDREI_LEQUIGAN_HOW TO USE ZOOM
LORRAINE ANDREI_LEQUIGAN_HOW TO USE ZOOMLORRAINE ANDREI_LEQUIGAN_HOW TO USE ZOOM
LORRAINE ANDREI_LEQUIGAN_HOW TO USE ZOOM
 
Artificia Intellicence and XPath Extension Functions
Artificia Intellicence and XPath Extension FunctionsArtificia Intellicence and XPath Extension Functions
Artificia Intellicence and XPath Extension Functions
 
Webinar On-Demand: Using Flutter for Embedded
Webinar On-Demand: Using Flutter for EmbeddedWebinar On-Demand: Using Flutter for Embedded
Webinar On-Demand: Using Flutter for Embedded
 
E-Invoicing Implementation: A Step-by-Step Guide for Saudi Arabian Companies
E-Invoicing Implementation: A Step-by-Step Guide for Saudi Arabian CompaniesE-Invoicing Implementation: A Step-by-Step Guide for Saudi Arabian Companies
E-Invoicing Implementation: A Step-by-Step Guide for Saudi Arabian Companies
 
How to write a program in any programming language
How to write a program in any programming languageHow to write a program in any programming language
How to write a program in any programming language
 
Measures in SQL (SIGMOD 2024, Santiago, Chile)
Measures in SQL (SIGMOD 2024, Santiago, Chile)Measures in SQL (SIGMOD 2024, Santiago, Chile)
Measures in SQL (SIGMOD 2024, Santiago, Chile)
 
E-commerce Development Services- Hornet Dynamics
E-commerce Development Services- Hornet DynamicsE-commerce Development Services- Hornet Dynamics
E-commerce Development Services- Hornet Dynamics
 
ALGIT - Assembly Line for Green IT - Numbers, Data, Facts
ALGIT - Assembly Line for Green IT - Numbers, Data, FactsALGIT - Assembly Line for Green IT - Numbers, Data, Facts
ALGIT - Assembly Line for Green IT - Numbers, Data, Facts
 
Introducing Crescat - Event Management Software for Venues, Festivals and Eve...
Introducing Crescat - Event Management Software for Venues, Festivals and Eve...Introducing Crescat - Event Management Software for Venues, Festivals and Eve...
Introducing Crescat - Event Management Software for Venues, Festivals and Eve...
 
Unveiling the Advantages of Agile Software Development.pdf
Unveiling the Advantages of Agile Software Development.pdfUnveiling the Advantages of Agile Software Development.pdf
Unveiling the Advantages of Agile Software Development.pdf
 
Top Benefits of Using Salesforce Healthcare CRM for Patient Management.pdf
Top Benefits of Using Salesforce Healthcare CRM for Patient Management.pdfTop Benefits of Using Salesforce Healthcare CRM for Patient Management.pdf
Top Benefits of Using Salesforce Healthcare CRM for Patient Management.pdf
 
socradar-q1-2024-aviation-industry-report.pdf
socradar-q1-2024-aviation-industry-report.pdfsocradar-q1-2024-aviation-industry-report.pdf
socradar-q1-2024-aviation-industry-report.pdf
 
Empowering Growth with Best Software Development Company in Noida - Deuglo
Empowering Growth with Best Software  Development Company in Noida - DeugloEmpowering Growth with Best Software  Development Company in Noida - Deuglo
Empowering Growth with Best Software Development Company in Noida - Deuglo
 
Transform Your Communication with Cloud-Based IVR Solutions
Transform Your Communication with Cloud-Based IVR SolutionsTransform Your Communication with Cloud-Based IVR Solutions
Transform Your Communication with Cloud-Based IVR Solutions
 
How Can Hiring A Mobile App Development Company Help Your Business Grow?
How Can Hiring A Mobile App Development Company Help Your Business Grow?How Can Hiring A Mobile App Development Company Help Your Business Grow?
How Can Hiring A Mobile App Development Company Help Your Business Grow?
 
Odoo ERP Vs. Traditional ERP Systems – A Comparative Analysis
Odoo ERP Vs. Traditional ERP Systems – A Comparative AnalysisOdoo ERP Vs. Traditional ERP Systems – A Comparative Analysis
Odoo ERP Vs. Traditional ERP Systems – A Comparative Analysis
 
Requirement Traceability in Xen Functional Safety
Requirement Traceability in Xen Functional SafetyRequirement Traceability in Xen Functional Safety
Requirement Traceability in Xen Functional Safety
 
Using Query Store in Azure PostgreSQL to Understand Query Performance
Using Query Store in Azure PostgreSQL to Understand Query PerformanceUsing Query Store in Azure PostgreSQL to Understand Query Performance
Using Query Store in Azure PostgreSQL to Understand Query Performance
 
Energy consumption of Database Management - Florina Jonuzi
Energy consumption of Database Management - Florina JonuziEnergy consumption of Database Management - Florina Jonuzi
Energy consumption of Database Management - Florina Jonuzi
 

Testing in the Age of Machine Learning

  • 1. @vinaya1980 @hrishikarekar TESTING IN THE AGE OF MACHINE LEARNING March 2016
  • 2. https://www.youtube.com/watch?v=DCgHsxISE0Q W H E R E A R E W E H E A D E D httpsweforum.org/agenda/2017/01/worried- about-ai-taking-your-job-its-already-happening-in- japan?utm_content=buffera14af&utm_medium=s ocial&utm_source=twitter.com&utm_campaign=b uffer
  • 3. THE WORLD OF ARTIFICIAL INTELLIGENCE NLP Neural Networks MACHINE LEARNING DEEP LEARNING …..
  • 4. THE FUTURE IS ALREADY HERE Google RankBrain Assistants
  • 5. HOW DOES IT WORK Supervised • Labelled data • Given new data, predict outcome • Classification Unsupervised • No labels • Find hidden structures • Clustering Reinforcement • Decision process • Actions are rewarded or punished • Learns to optimize rewards
  • 7. SUPERVISED LEARNING - CLASSIFCATION http://www.mikedeff.in/MLIntro.PNG
  • 8. INTO THE REALM OF PROBABILITIES Y = f ( x ) Y ≈ f ( x ) What is scrum ? { "Prediction": { "details": { "Algorithm": "SGD", "PredictiveModelType": "MULTICLASS" }, "predictedLabel": "definition", "predictedScores": { "advantages": 0.0001860455668065697, "characteristics": 0.00006915141420904547, "compare": 0.00017757616296876222, "definition": 0.9970965385437012, "disadvantages": 0.0000534967657586094, } } } Can you tell me about scrum ? { "Prediction": { "details": { "Algorithm": "SGD", "PredictiveModelType": "MULTICLASS" }, "predictedLabel": "definition", "predictedScores": { "advantages": 0.01977257989346981, "characteristics": 0.022757112979888916, "compare": 0.008386141620576382, "definition": 0.21092116832733154, "disadvantages": 0.04002799838781357 } } }
  • 9. TOLERANCE LEVELS Y ≈ f ( x ) Know the probability that is within acceptable limits
  • 10. EVALUATE WITH DIFFERENT MODELS Evaluate against a set of algorithms to iterate towards a model that’s closest representation and for further tuning https://s3.amazonaws.com/MLMastery/MachineLearningAlgorithms.png?__s=h4reg8jqwyg4sz3bzdqf
  • 11. EVALUATION – DATA SET APPROACHES Random split • 70% train, 30% test K-fold cross validation Split into 3 datasets • #1 Train on 1 and 2, test on 3 • #2 Train on 2 and 3, test on 1 • #3 Train on 1 and 3, test on 2 Never use the same dataset for training and evaluating
  • 12. ITERATIVE – LEARNING PROCESS Be prepared to throw the model and start again 
  • 13. MODEL ACCURACY - CONFUSION MATRIX Strive for better models, not 100% accurate.
  • 15. MODEL IS AS GOOD AS THE TRAINING DATA If all of the algorithms perform poorly, • it maybe worth considering if there is a lack of learning structure in the data set • some transformation needed to make the structure more learnable • remove unnecessary noise - stop words are typically removed because they cause unnecessary noise)
  • 16. SUMMARY Machine Learning applications demand a shift in testing approach • Use objective acceptance levels to evaluate the application • Express test outcomes in statistical terms • Have a high level understanding of the underlying working of the application