SlideShare a Scribd company logo
25 – August – 16
Test Selection
with Moose In Industry:
Impact of Granularity
Vincent Blondeau
Vincent Blondeau | 25 - August - 16 | 2
Context
▶ Industrial PhD in a major international IT company
–  +7 000 employees
–  17 countries
–  Problems from the field
Vincent Blondeau | 25 - August - 16 | 3
Test Case Selection After a Change
?
Test All
Takes a long time
(3 hours)
Test Nothing
Not a solution
Vincent Blondeau | 25 - August - 16 | 4
Test Case Selection After a Change
▶ Question
–  Does test selection modify developers’ habits and
enhance software quality?
▶ Goal
–  Save time and improve quality
–  Select tests to relaunch after a change in the source
code
–  Any kind of test: End to end, performance…
▶ Comparison of approaches on real changes
–  Call graph analysis by static approach
–  Comparison with dynamic approach
Vincent Blondeau | 25 - August - 16 | 5
Two approaches
▶ Static Approach
–  Create a model of the system with Moose
–  Navigate the call graph from a changed source code
to find the tests
Properties
–  Allows to find multiple tracks to the changed source
–  No test execution
Vincent Blondeau | 25 - August - 16 | 6
Two approaches
▶ Dynamic Approach
–  Execute the tests
•  Map the tests to the covered code
–  Relaunch the tests related to changed source code
Properties
–  Dependent to the test data
–  The tests have to be executed
Vincent Blondeau | 25 - August - 16 | 7
Issue Classification
▶ Problems in test selection approaches arise when
there is a break in the dependency graph representing
the system.
–  Third-party breaks
–  Multi-program breaks
–  Dynamic breaks
–  Polymorphism breaks
Vincent Blondeau | 25 - August - 16 | 8
Experiment
▶ Hypothesis: Dynamic approach is the oracle
–  With some flaws:
•  Does not work on failing or in error tests
•  Requires time to be performed
▶ Approach
–  Compare influence of real source code changes
–  Simulate code change on several existing projects
Vincent Blondeau | 25 - August - 16 | 9
Experiment
▶ Consider real commits by mining repositories
–  Weight each covered method by the number of
commits
–  Group covered methods in commits
•  Considered real method commit groups
Vincent Blondeau | 25 - August - 16 | 10
Projects: Metrics
Metric P1 P2 P3
KLOC Core 447 716 302
# Green Tests 5 323 168 3 035
# Total Methods 9 808 56 661 45 671
# Methods Covered
4720
(48%)
3 261
(6%)
8 143
(18%)
#Commits 2 217 467 2 115
Avg Methods/Commit 24 129 37
Avg Files/Commit 7 18 17
Vincent Blondeau | 25 - August - 16 | 11
Metrics
▶ Number of selected tests
–  Ratio of the total test suite to relaunch
▶ Precision
–  How many selected tests are relevant?
▶ Recall
–  How many relevant tests are selected?
Vincent Blondeau | 25 - August - 16 | 12
#Selected Tests Precision Recall
1 Meth. Weig. 1 Meth. Weig. 1 Meth. Weig.
P1 3% 3%
43%
42% 91%
92%
P2 0.8%
1% 61% 64%
59% 62%
P3 2% 2%
34% 41%
39%33%
Weighting of methods with the
number of commits
Vincent Blondeau | 25 - August - 16 | 13
#Selected Tests Precision Recall
1 Meth. Commit 1 Meth. Commit 1 Meth. Commit
P1
3%
4%
43%
55% 91%
81%
P2
0.8%
3% 61% 64%
45% 45%
P3
2%
6% 49%
41%
56%
34%
Methods grouped in commits
Vincent Blondeau | 25 - August - 16 | 14
Conclusion
▶ Considering commits instead of individual methods
tends to worsen the results
▶ Impact on projects is different
▶ Low ratio of selected tests, so still acceptable
Future steps
▶ Better understand how tests are used by developers
▶ Provide a tool for developers to select tests
Worldline is a registered trademark of Atos Worldline SAS. June 2013
© 2013 Atos. Confidential information owned by Atos Worldline, to be
used by the recipient only. This document, or any part of it, may not
be reproduced, copied, circulated and/or distributed nor quoted
without prior written approval from Atos Worldline.
25-August-16
Thanks!

More Related Content

What's hot

Test Progress Monitoring and Control
Test Progress Monitoring and ControlTest Progress Monitoring and Control
Test Progress Monitoring and Control
NolaCita
 
Metrics
MetricsMetrics
Report_Card March 2016
Report_Card March 2016Report_Card March 2016
Report_Card March 2016
Carrie E. Williams
 
ICST/NEXTA2020: Experience Report on Test Automation Process Improvement
ICST/NEXTA2020: Experience Report on Test Automation Process ImprovementICST/NEXTA2020: Experience Report on Test Automation Process Improvement
ICST/NEXTA2020: Experience Report on Test Automation Process Improvement
Maaret Pyhäjärvi
 
Combining Requirements Engineering and Testing, QA&TEST Bilbao 2013
Combining Requirements Engineering and Testing, QA&TEST Bilbao 2013Combining Requirements Engineering and Testing, QA&TEST Bilbao 2013
Combining Requirements Engineering and Testing, QA&TEST Bilbao 2013
SYSQA BV
 
Comparison Of Pre And Post Assessments Astronomy
Comparison Of Pre And Post Assessments AstronomyComparison Of Pre And Post Assessments Astronomy
Comparison Of Pre And Post Assessments Astronomy
Staci Trekles
 
Test automation implementation strategy
Test automation implementation strategy Test automation implementation strategy
Test automation implementation strategy
Forte Group
 
Assessment 3 descriptor
Assessment 3 descriptorAssessment 3 descriptor
Assessment 3 descriptor
Praneel Chand
 
An overview of management & law
An overview of management & lawAn overview of management & law
An overview of management & law
nareshbro
 
Report_Card September
Report_Card SeptemberReport_Card September
Report_Card September
Carrie E. Williams
 
Advanced Features of Selenium Automated Web Testing
Advanced Features of Selenium Automated Web TestingAdvanced Features of Selenium Automated Web Testing
Advanced Features of Selenium Automated Web Testing
Michael Hughes
 
Report_Card October 2015
Report_Card October 2015Report_Card October 2015
Report_Card October 2015
Carrie E. Williams
 
Implementing Outcome Budget_Key Concepts_20Feb
Implementing Outcome Budget_Key Concepts_20FebImplementing Outcome Budget_Key Concepts_20Feb
Implementing Outcome Budget_Key Concepts_20Feb
Anant Mittal
 
Report_Card 1st Qtr 2016
Report_Card  1st Qtr 2016Report_Card  1st Qtr 2016
Report_Card 1st Qtr 2016
Carrie E. Williams
 

What's hot (14)

Test Progress Monitoring and Control
Test Progress Monitoring and ControlTest Progress Monitoring and Control
Test Progress Monitoring and Control
 
Metrics
MetricsMetrics
Metrics
 
Report_Card March 2016
Report_Card March 2016Report_Card March 2016
Report_Card March 2016
 
ICST/NEXTA2020: Experience Report on Test Automation Process Improvement
ICST/NEXTA2020: Experience Report on Test Automation Process ImprovementICST/NEXTA2020: Experience Report on Test Automation Process Improvement
ICST/NEXTA2020: Experience Report on Test Automation Process Improvement
 
Combining Requirements Engineering and Testing, QA&TEST Bilbao 2013
Combining Requirements Engineering and Testing, QA&TEST Bilbao 2013Combining Requirements Engineering and Testing, QA&TEST Bilbao 2013
Combining Requirements Engineering and Testing, QA&TEST Bilbao 2013
 
Comparison Of Pre And Post Assessments Astronomy
Comparison Of Pre And Post Assessments AstronomyComparison Of Pre And Post Assessments Astronomy
Comparison Of Pre And Post Assessments Astronomy
 
Test automation implementation strategy
Test automation implementation strategy Test automation implementation strategy
Test automation implementation strategy
 
Assessment 3 descriptor
Assessment 3 descriptorAssessment 3 descriptor
Assessment 3 descriptor
 
An overview of management & law
An overview of management & lawAn overview of management & law
An overview of management & law
 
Report_Card September
Report_Card SeptemberReport_Card September
Report_Card September
 
Advanced Features of Selenium Automated Web Testing
Advanced Features of Selenium Automated Web TestingAdvanced Features of Selenium Automated Web Testing
Advanced Features of Selenium Automated Web Testing
 
Report_Card October 2015
Report_Card October 2015Report_Card October 2015
Report_Card October 2015
 
Implementing Outcome Budget_Key Concepts_20Feb
Implementing Outcome Budget_Key Concepts_20FebImplementing Outcome Budget_Key Concepts_20Feb
Implementing Outcome Budget_Key Concepts_20Feb
 
Report_Card 1st Qtr 2016
Report_Card  1st Qtr 2016Report_Card  1st Qtr 2016
Report_Card 1st Qtr 2016
 

Viewers also liked

Bloc: a Modern Core for Highly Dynamic Graphics
Bloc: a Modern Core for Highly Dynamic Graphics Bloc: a Modern Core for Highly Dynamic Graphics
Bloc: a Modern Core for Highly Dynamic Graphics
ESUG
 
New Native Windows UI possibilities in Cincom Smalltalk ObjectStudio
New Native Windows UI possibilities in Cincom Smalltalk ObjectStudioNew Native Windows UI possibilities in Cincom Smalltalk ObjectStudio
New Native Windows UI possibilities in Cincom Smalltalk ObjectStudio
ESUG
 
Prototyping Software Product Lines analysis with Pharo
Prototyping Software Product Lines analysis with PharoPrototyping Software Product Lines analysis with Pharo
Prototyping Software Product Lines analysis with Pharo
ESUG
 
Tugrik: A new persistence option for Pharo
Tugrik: A new persistence option for PharoTugrik: A new persistence option for Pharo
Tugrik: A new persistence option for Pharo
ESUG
 
Pillar: one language for all supports
Pillar: one language for all supportsPillar: one language for all supports
Pillar: one language for all supports
ESUG
 
The Glamorous Toolkit: Towards a novel live IDE
The Glamorous Toolkit: Towards a novel live IDEThe Glamorous Toolkit: Towards a novel live IDE
The Glamorous Toolkit: Towards a novel live IDE
ESUG
 
Lub: a DSL for Dynamic Context Oriented Programming
Lub: a DSL for Dynamic Context Oriented ProgrammingLub: a DSL for Dynamic Context Oriented Programming
Lub: a DSL for Dynamic Context Oriented Programming
ESUG
 
seamless – Object Oriented CMS System
seamless – Object Oriented CMS Systemseamless – Object Oriented CMS System
seamless – Object Oriented CMS System
ESUG
 

Viewers also liked (8)

Bloc: a Modern Core for Highly Dynamic Graphics
Bloc: a Modern Core for Highly Dynamic Graphics Bloc: a Modern Core for Highly Dynamic Graphics
Bloc: a Modern Core for Highly Dynamic Graphics
 
New Native Windows UI possibilities in Cincom Smalltalk ObjectStudio
New Native Windows UI possibilities in Cincom Smalltalk ObjectStudioNew Native Windows UI possibilities in Cincom Smalltalk ObjectStudio
New Native Windows UI possibilities in Cincom Smalltalk ObjectStudio
 
Prototyping Software Product Lines analysis with Pharo
Prototyping Software Product Lines analysis with PharoPrototyping Software Product Lines analysis with Pharo
Prototyping Software Product Lines analysis with Pharo
 
Tugrik: A new persistence option for Pharo
Tugrik: A new persistence option for PharoTugrik: A new persistence option for Pharo
Tugrik: A new persistence option for Pharo
 
Pillar: one language for all supports
Pillar: one language for all supportsPillar: one language for all supports
Pillar: one language for all supports
 
The Glamorous Toolkit: Towards a novel live IDE
The Glamorous Toolkit: Towards a novel live IDEThe Glamorous Toolkit: Towards a novel live IDE
The Glamorous Toolkit: Towards a novel live IDE
 
Lub: a DSL for Dynamic Context Oriented Programming
Lub: a DSL for Dynamic Context Oriented ProgrammingLub: a DSL for Dynamic Context Oriented Programming
Lub: a DSL for Dynamic Context Oriented Programming
 
seamless – Object Oriented CMS System
seamless – Object Oriented CMS Systemseamless – Object Oriented CMS System
seamless – Object Oriented CMS System
 

Similar to Test Selection with Moose In Industry - Impact of Granularity

Analysis of food and drinks manufacturing companies
Analysis of food and drinks manufacturing companiesAnalysis of food and drinks manufacturing companies
Analysis of food and drinks manufacturing companies
David Fearnley-Brown
 
business review
business reviewbusiness review
business review
David Fearnley-Brown
 
Day1 1620-1705-maple-pranabendubhattacharyya-131008043643-phpapp02
Day1 1620-1705-maple-pranabendubhattacharyya-131008043643-phpapp02Day1 1620-1705-maple-pranabendubhattacharyya-131008043643-phpapp02
Day1 1620-1705-maple-pranabendubhattacharyya-131008043643-phpapp02
PMI_IREP_TP
 
Day 1 1620 - 1705 - maple - pranabendu bhattacharyya
Day 1   1620 - 1705 - maple - pranabendu bhattacharyyaDay 1   1620 - 1705 - maple - pranabendu bhattacharyya
Day 1 1620 - 1705 - maple - pranabendu bhattacharyya
PMI2011
 
Use the Windshield, Not the Mirror Predictive Metrics that Drive Successful ...
 Use the Windshield, Not the Mirror Predictive Metrics that Drive Successful ... Use the Windshield, Not the Mirror Predictive Metrics that Drive Successful ...
Use the Windshield, Not the Mirror Predictive Metrics that Drive Successful ...
Seapine Software
 
A Framework for Sustainability Assessment Technologies(SAT)
A Framework for Sustainability Assessment Technologies(SAT)A Framework for Sustainability Assessment Technologies(SAT)
A Framework for Sustainability Assessment Technologies(SAT)
Ekonnect Knowledge Foundation
 
Continuous testing in agile projects 2015
Continuous testing in agile projects 2015Continuous testing in agile projects 2015
Continuous testing in agile projects 2015
Fabricio Epaminondas
 
Towards a systems approach in Ecodesign and Energy Labelling: How to make the...
Towards a systems approach in Ecodesign and Energy Labelling: How to make the...Towards a systems approach in Ecodesign and Energy Labelling: How to make the...
Towards a systems approach in Ecodesign and Energy Labelling: How to make the...
Leonardo ENERGY
 
ARF foq2 Day Router Presentation
ARF foq2 Day Router Presentation ARF foq2 Day Router Presentation
ARF foq2 Day Router Presentation
Federated Sample
 
Making Right Choices: Sustainability Assessment of Technologies
Making Right Choices: Sustainability Assessment of TechnologiesMaking Right Choices: Sustainability Assessment of Technologies
Making Right Choices: Sustainability Assessment of Technologies
Prasad Modak
 
Performance Testing in the Agile Lifecycle
Performance Testing in the Agile LifecyclePerformance Testing in the Agile Lifecycle
Performance Testing in the Agile Lifecycle
TechWell
 
PEX Week: iDatix Workshop Part 3
PEX Week: iDatix Workshop Part 3PEX Week: iDatix Workshop Part 3
PEX Week: iDatix Workshop Part 3
iDatix
 
White paper on Quality by Design
White paper on Quality by DesignWhite paper on Quality by Design
White paper on Quality by Design
Bir Gujral PhD, MBA
 
ISTQBCH2.ppt
ISTQBCH2.pptISTQBCH2.ppt
ISTQBCH2.ppt
RppsKumar1
 
ISTQBCH2.ppt
ISTQBCH2.pptISTQBCH2.ppt
ISTQBCH2.ppt
ghkadous
 
Prototyping
PrototypingPrototyping
Prototyping
Ifa Laili
 
Ph.D. Research Update: Year#3 Annual Progress and Planned Activities
Ph.D. Research Update: Year#3 Annual Progress and Planned ActivitiesPh.D. Research Update: Year#3 Annual Progress and Planned Activities
Ph.D. Research Update: Year#3 Annual Progress and Planned Activities
Lighton Phiri
 
Software metrics to predict the health of a project? - An assessment in a maj...
Software metrics to predict the health of a project? - An assessment in a maj...Software metrics to predict the health of a project? - An assessment in a maj...
Software metrics to predict the health of a project? - An assessment in a maj...
ESUG
 
Sqcm
SqcmSqcm
Toolsusa01.ppt
Toolsusa01.pptToolsusa01.ppt
Toolsusa01.ppt
Yann-Gaël Guéhéneuc
 

Similar to Test Selection with Moose In Industry - Impact of Granularity (20)

Analysis of food and drinks manufacturing companies
Analysis of food and drinks manufacturing companiesAnalysis of food and drinks manufacturing companies
Analysis of food and drinks manufacturing companies
 
business review
business reviewbusiness review
business review
 
Day1 1620-1705-maple-pranabendubhattacharyya-131008043643-phpapp02
Day1 1620-1705-maple-pranabendubhattacharyya-131008043643-phpapp02Day1 1620-1705-maple-pranabendubhattacharyya-131008043643-phpapp02
Day1 1620-1705-maple-pranabendubhattacharyya-131008043643-phpapp02
 
Day 1 1620 - 1705 - maple - pranabendu bhattacharyya
Day 1   1620 - 1705 - maple - pranabendu bhattacharyyaDay 1   1620 - 1705 - maple - pranabendu bhattacharyya
Day 1 1620 - 1705 - maple - pranabendu bhattacharyya
 
Use the Windshield, Not the Mirror Predictive Metrics that Drive Successful ...
 Use the Windshield, Not the Mirror Predictive Metrics that Drive Successful ... Use the Windshield, Not the Mirror Predictive Metrics that Drive Successful ...
Use the Windshield, Not the Mirror Predictive Metrics that Drive Successful ...
 
A Framework for Sustainability Assessment Technologies(SAT)
A Framework for Sustainability Assessment Technologies(SAT)A Framework for Sustainability Assessment Technologies(SAT)
A Framework for Sustainability Assessment Technologies(SAT)
 
Continuous testing in agile projects 2015
Continuous testing in agile projects 2015Continuous testing in agile projects 2015
Continuous testing in agile projects 2015
 
Towards a systems approach in Ecodesign and Energy Labelling: How to make the...
Towards a systems approach in Ecodesign and Energy Labelling: How to make the...Towards a systems approach in Ecodesign and Energy Labelling: How to make the...
Towards a systems approach in Ecodesign and Energy Labelling: How to make the...
 
ARF foq2 Day Router Presentation
ARF foq2 Day Router Presentation ARF foq2 Day Router Presentation
ARF foq2 Day Router Presentation
 
Making Right Choices: Sustainability Assessment of Technologies
Making Right Choices: Sustainability Assessment of TechnologiesMaking Right Choices: Sustainability Assessment of Technologies
Making Right Choices: Sustainability Assessment of Technologies
 
Performance Testing in the Agile Lifecycle
Performance Testing in the Agile LifecyclePerformance Testing in the Agile Lifecycle
Performance Testing in the Agile Lifecycle
 
PEX Week: iDatix Workshop Part 3
PEX Week: iDatix Workshop Part 3PEX Week: iDatix Workshop Part 3
PEX Week: iDatix Workshop Part 3
 
White paper on Quality by Design
White paper on Quality by DesignWhite paper on Quality by Design
White paper on Quality by Design
 
ISTQBCH2.ppt
ISTQBCH2.pptISTQBCH2.ppt
ISTQBCH2.ppt
 
ISTQBCH2.ppt
ISTQBCH2.pptISTQBCH2.ppt
ISTQBCH2.ppt
 
Prototyping
PrototypingPrototyping
Prototyping
 
Ph.D. Research Update: Year#3 Annual Progress and Planned Activities
Ph.D. Research Update: Year#3 Annual Progress and Planned ActivitiesPh.D. Research Update: Year#3 Annual Progress and Planned Activities
Ph.D. Research Update: Year#3 Annual Progress and Planned Activities
 
Software metrics to predict the health of a project? - An assessment in a maj...
Software metrics to predict the health of a project? - An assessment in a maj...Software metrics to predict the health of a project? - An assessment in a maj...
Software metrics to predict the health of a project? - An assessment in a maj...
 
Sqcm
SqcmSqcm
Sqcm
 
Toolsusa01.ppt
Toolsusa01.pptToolsusa01.ppt
Toolsusa01.ppt
 

More from ESUG

Workshop: Identifying concept inventories in agile programming
Workshop: Identifying concept inventories in agile programmingWorkshop: Identifying concept inventories in agile programming
Workshop: Identifying concept inventories in agile programming
ESUG
 
Technical documentation support in Pharo
Technical documentation support in PharoTechnical documentation support in Pharo
Technical documentation support in Pharo
ESUG
 
The Pharo Debugger and Debugging tools: Advances and Roadmap
The Pharo Debugger and Debugging tools: Advances and RoadmapThe Pharo Debugger and Debugging tools: Advances and Roadmap
The Pharo Debugger and Debugging tools: Advances and Roadmap
ESUG
 
Sequence: Pipeline modelling in Pharo
Sequence: Pipeline modelling in PharoSequence: Pipeline modelling in Pharo
Sequence: Pipeline modelling in Pharo
ESUG
 
Migration process from monolithic to micro frontend architecture in mobile ap...
Migration process from monolithic to micro frontend architecture in mobile ap...Migration process from monolithic to micro frontend architecture in mobile ap...
Migration process from monolithic to micro frontend architecture in mobile ap...
ESUG
 
Analyzing Dart Language with Pharo: Report and early results
Analyzing Dart Language with Pharo: Report and early resultsAnalyzing Dart Language with Pharo: Report and early results
Analyzing Dart Language with Pharo: Report and early results
ESUG
 
Transpiling Pharo Classes to JS ECMAScript 5 versus ECMAScript 6
Transpiling Pharo Classes to JS ECMAScript 5 versus ECMAScript 6Transpiling Pharo Classes to JS ECMAScript 5 versus ECMAScript 6
Transpiling Pharo Classes to JS ECMAScript 5 versus ECMAScript 6
ESUG
 
A Unit Test Metamodel for Test Generation
A Unit Test Metamodel for Test GenerationA Unit Test Metamodel for Test Generation
A Unit Test Metamodel for Test Generation
ESUG
 
Creating Unit Tests Using Genetic Programming
Creating Unit Tests Using Genetic ProgrammingCreating Unit Tests Using Genetic Programming
Creating Unit Tests Using Genetic Programming
ESUG
 
Threaded-Execution and CPS Provide Smooth Switching Between Execution Modes
Threaded-Execution and CPS Provide Smooth Switching Between Execution ModesThreaded-Execution and CPS Provide Smooth Switching Between Execution Modes
Threaded-Execution and CPS Provide Smooth Switching Between Execution Modes
ESUG
 
Exploring GitHub Actions through EGAD: An Experience Report
Exploring GitHub Actions through EGAD: An Experience ReportExploring GitHub Actions through EGAD: An Experience Report
Exploring GitHub Actions through EGAD: An Experience Report
ESUG
 
Pharo: a reflective language A first systematic analysis of reflective APIs
Pharo: a reflective language A first systematic analysis of reflective APIsPharo: a reflective language A first systematic analysis of reflective APIs
Pharo: a reflective language A first systematic analysis of reflective APIs
ESUG
 
Garbage Collector Tuning
Garbage Collector TuningGarbage Collector Tuning
Garbage Collector Tuning
ESUG
 
Improving Performance Through Object Lifetime Profiling: the DataFrame Case
Improving Performance Through Object Lifetime Profiling: the DataFrame CaseImproving Performance Through Object Lifetime Profiling: the DataFrame Case
Improving Performance Through Object Lifetime Profiling: the DataFrame Case
ESUG
 
Pharo DataFrame: Past, Present, and Future
Pharo DataFrame: Past, Present, and FuturePharo DataFrame: Past, Present, and Future
Pharo DataFrame: Past, Present, and Future
ESUG
 
thisContext in the Debugger
thisContext in the DebuggerthisContext in the Debugger
thisContext in the Debugger
ESUG
 
Websockets for Fencing Score
Websockets for Fencing ScoreWebsockets for Fencing Score
Websockets for Fencing Score
ESUG
 
ShowUs: PharoJS.org Develop in Pharo, Run on JavaScript
ShowUs: PharoJS.org Develop in Pharo, Run on JavaScriptShowUs: PharoJS.org Develop in Pharo, Run on JavaScript
ShowUs: PharoJS.org Develop in Pharo, Run on JavaScript
ESUG
 
Advanced Object- Oriented Design Mooc
Advanced Object- Oriented Design MoocAdvanced Object- Oriented Design Mooc
Advanced Object- Oriented Design Mooc
ESUG
 
A New Architecture Reconciling Refactorings and Transformations
A New Architecture Reconciling Refactorings and TransformationsA New Architecture Reconciling Refactorings and Transformations
A New Architecture Reconciling Refactorings and Transformations
ESUG
 

More from ESUG (20)

Workshop: Identifying concept inventories in agile programming
Workshop: Identifying concept inventories in agile programmingWorkshop: Identifying concept inventories in agile programming
Workshop: Identifying concept inventories in agile programming
 
Technical documentation support in Pharo
Technical documentation support in PharoTechnical documentation support in Pharo
Technical documentation support in Pharo
 
The Pharo Debugger and Debugging tools: Advances and Roadmap
The Pharo Debugger and Debugging tools: Advances and RoadmapThe Pharo Debugger and Debugging tools: Advances and Roadmap
The Pharo Debugger and Debugging tools: Advances and Roadmap
 
Sequence: Pipeline modelling in Pharo
Sequence: Pipeline modelling in PharoSequence: Pipeline modelling in Pharo
Sequence: Pipeline modelling in Pharo
 
Migration process from monolithic to micro frontend architecture in mobile ap...
Migration process from monolithic to micro frontend architecture in mobile ap...Migration process from monolithic to micro frontend architecture in mobile ap...
Migration process from monolithic to micro frontend architecture in mobile ap...
 
Analyzing Dart Language with Pharo: Report and early results
Analyzing Dart Language with Pharo: Report and early resultsAnalyzing Dart Language with Pharo: Report and early results
Analyzing Dart Language with Pharo: Report and early results
 
Transpiling Pharo Classes to JS ECMAScript 5 versus ECMAScript 6
Transpiling Pharo Classes to JS ECMAScript 5 versus ECMAScript 6Transpiling Pharo Classes to JS ECMAScript 5 versus ECMAScript 6
Transpiling Pharo Classes to JS ECMAScript 5 versus ECMAScript 6
 
A Unit Test Metamodel for Test Generation
A Unit Test Metamodel for Test GenerationA Unit Test Metamodel for Test Generation
A Unit Test Metamodel for Test Generation
 
Creating Unit Tests Using Genetic Programming
Creating Unit Tests Using Genetic ProgrammingCreating Unit Tests Using Genetic Programming
Creating Unit Tests Using Genetic Programming
 
Threaded-Execution and CPS Provide Smooth Switching Between Execution Modes
Threaded-Execution and CPS Provide Smooth Switching Between Execution ModesThreaded-Execution and CPS Provide Smooth Switching Between Execution Modes
Threaded-Execution and CPS Provide Smooth Switching Between Execution Modes
 
Exploring GitHub Actions through EGAD: An Experience Report
Exploring GitHub Actions through EGAD: An Experience ReportExploring GitHub Actions through EGAD: An Experience Report
Exploring GitHub Actions through EGAD: An Experience Report
 
Pharo: a reflective language A first systematic analysis of reflective APIs
Pharo: a reflective language A first systematic analysis of reflective APIsPharo: a reflective language A first systematic analysis of reflective APIs
Pharo: a reflective language A first systematic analysis of reflective APIs
 
Garbage Collector Tuning
Garbage Collector TuningGarbage Collector Tuning
Garbage Collector Tuning
 
Improving Performance Through Object Lifetime Profiling: the DataFrame Case
Improving Performance Through Object Lifetime Profiling: the DataFrame CaseImproving Performance Through Object Lifetime Profiling: the DataFrame Case
Improving Performance Through Object Lifetime Profiling: the DataFrame Case
 
Pharo DataFrame: Past, Present, and Future
Pharo DataFrame: Past, Present, and FuturePharo DataFrame: Past, Present, and Future
Pharo DataFrame: Past, Present, and Future
 
thisContext in the Debugger
thisContext in the DebuggerthisContext in the Debugger
thisContext in the Debugger
 
Websockets for Fencing Score
Websockets for Fencing ScoreWebsockets for Fencing Score
Websockets for Fencing Score
 
ShowUs: PharoJS.org Develop in Pharo, Run on JavaScript
ShowUs: PharoJS.org Develop in Pharo, Run on JavaScriptShowUs: PharoJS.org Develop in Pharo, Run on JavaScript
ShowUs: PharoJS.org Develop in Pharo, Run on JavaScript
 
Advanced Object- Oriented Design Mooc
Advanced Object- Oriented Design MoocAdvanced Object- Oriented Design Mooc
Advanced Object- Oriented Design Mooc
 
A New Architecture Reconciling Refactorings and Transformations
A New Architecture Reconciling Refactorings and TransformationsA New Architecture Reconciling Refactorings and Transformations
A New Architecture Reconciling Refactorings and Transformations
 

Recently uploaded

一比一原版(sdsu毕业证书)圣地亚哥州立大学毕业证如何办理
一比一原版(sdsu毕业证书)圣地亚哥州立大学毕业证如何办理一比一原版(sdsu毕业证书)圣地亚哥州立大学毕业证如何办理
一比一原版(sdsu毕业证书)圣地亚哥州立大学毕业证如何办理
kgyxske
 
E-Invoicing Implementation: A Step-by-Step Guide for Saudi Arabian Companies
E-Invoicing Implementation: A Step-by-Step Guide for Saudi Arabian CompaniesE-Invoicing Implementation: A Step-by-Step Guide for Saudi Arabian Companies
E-Invoicing Implementation: A Step-by-Step Guide for Saudi Arabian Companies
Quickdice ERP
 
Preparing Non - Technical Founders for Engaging a Tech Agency
Preparing Non - Technical Founders for Engaging  a  Tech AgencyPreparing Non - Technical Founders for Engaging  a  Tech Agency
Preparing Non - Technical Founders for Engaging a Tech Agency
ISH Technologies
 
WWDC 2024 Keynote Review: For CocoaCoders Austin
WWDC 2024 Keynote Review: For CocoaCoders AustinWWDC 2024 Keynote Review: For CocoaCoders Austin
WWDC 2024 Keynote Review: For CocoaCoders Austin
Patrick Weigel
 
Project Management: The Role of Project Dashboards.pdf
Project Management: The Role of Project Dashboards.pdfProject Management: The Role of Project Dashboards.pdf
Project Management: The Role of Project Dashboards.pdf
Karya Keeper
 
Baha Majid WCA4Z IBM Z Customer Council Boston June 2024.pdf
Baha Majid WCA4Z IBM Z Customer Council Boston June 2024.pdfBaha Majid WCA4Z IBM Z Customer Council Boston June 2024.pdf
Baha Majid WCA4Z IBM Z Customer Council Boston June 2024.pdf
Baha Majid
 
Unlock the Secrets to Effortless Video Creation with Invideo: Your Ultimate G...
Unlock the Secrets to Effortless Video Creation with Invideo: Your Ultimate G...Unlock the Secrets to Effortless Video Creation with Invideo: Your Ultimate G...
Unlock the Secrets to Effortless Video Creation with Invideo: Your Ultimate G...
The Third Creative Media
 
如何办理(hull学位证书)英国赫尔大学毕业证硕士文凭原版一模一样
如何办理(hull学位证书)英国赫尔大学毕业证硕士文凭原版一模一样如何办理(hull学位证书)英国赫尔大学毕业证硕士文凭原版一模一样
如何办理(hull学位证书)英国赫尔大学毕业证硕士文凭原版一模一样
gapen1
 
Why Apache Kafka Clusters Are Like Galaxies (And Other Cosmic Kafka Quandarie...
Why Apache Kafka Clusters Are Like Galaxies (And Other Cosmic Kafka Quandarie...Why Apache Kafka Clusters Are Like Galaxies (And Other Cosmic Kafka Quandarie...
Why Apache Kafka Clusters Are Like Galaxies (And Other Cosmic Kafka Quandarie...
Paul Brebner
 
Migration From CH 1.0 to CH 2.0 and Mule 4.6 & Java 17 Upgrade.pptx
Migration From CH 1.0 to CH 2.0 and  Mule 4.6 & Java 17 Upgrade.pptxMigration From CH 1.0 to CH 2.0 and  Mule 4.6 & Java 17 Upgrade.pptx
Migration From CH 1.0 to CH 2.0 and Mule 4.6 & Java 17 Upgrade.pptx
ervikas4
 
Mobile App Development Company In Noida | Drona Infotech
Mobile App Development Company In Noida | Drona InfotechMobile App Development Company In Noida | Drona Infotech
Mobile App Development Company In Noida | Drona Infotech
Drona Infotech
 
ALGIT - Assembly Line for Green IT - Numbers, Data, Facts
ALGIT - Assembly Line for Green IT - Numbers, Data, FactsALGIT - Assembly Line for Green IT - Numbers, Data, Facts
ALGIT - Assembly Line for Green IT - Numbers, Data, Facts
Green Software Development
 
ppt on the brain chip neuralink.pptx
ppt  on   the brain  chip neuralink.pptxppt  on   the brain  chip neuralink.pptx
ppt on the brain chip neuralink.pptx
Reetu63
 
Safelyio Toolbox Talk Softwate & App (How To Digitize Safety Meetings)
Safelyio Toolbox Talk Softwate & App (How To Digitize Safety Meetings)Safelyio Toolbox Talk Softwate & App (How To Digitize Safety Meetings)
Safelyio Toolbox Talk Softwate & App (How To Digitize Safety Meetings)
safelyiotech
 
Everything You Need to Know About X-Sign: The eSign Functionality of XfilesPr...
Everything You Need to Know About X-Sign: The eSign Functionality of XfilesPr...Everything You Need to Know About X-Sign: The eSign Functionality of XfilesPr...
Everything You Need to Know About X-Sign: The eSign Functionality of XfilesPr...
XfilesPro
 
ACE - Team 24 Wrapup event at ahmedabad.
ACE - Team 24 Wrapup event at ahmedabad.ACE - Team 24 Wrapup event at ahmedabad.
ACE - Team 24 Wrapup event at ahmedabad.
Maitrey Patel
 
Unveiling the Advantages of Agile Software Development.pdf
Unveiling the Advantages of Agile Software Development.pdfUnveiling the Advantages of Agile Software Development.pdf
Unveiling the Advantages of Agile Software Development.pdf
brainerhub1
 
INTRODUCTION TO AI CLASSICAL THEORY TARGETED EXAMPLES
INTRODUCTION TO AI CLASSICAL THEORY TARGETED EXAMPLESINTRODUCTION TO AI CLASSICAL THEORY TARGETED EXAMPLES
INTRODUCTION TO AI CLASSICAL THEORY TARGETED EXAMPLES
anfaltahir1010
 
Microservice Teams - How the cloud changes the way we work
Microservice Teams - How the cloud changes the way we workMicroservice Teams - How the cloud changes the way we work
Microservice Teams - How the cloud changes the way we work
Sven Peters
 
UI5con 2024 - Boost Your Development Experience with UI5 Tooling Extensions
UI5con 2024 - Boost Your Development Experience with UI5 Tooling ExtensionsUI5con 2024 - Boost Your Development Experience with UI5 Tooling Extensions
UI5con 2024 - Boost Your Development Experience with UI5 Tooling Extensions
Peter Muessig
 

Recently uploaded (20)

一比一原版(sdsu毕业证书)圣地亚哥州立大学毕业证如何办理
一比一原版(sdsu毕业证书)圣地亚哥州立大学毕业证如何办理一比一原版(sdsu毕业证书)圣地亚哥州立大学毕业证如何办理
一比一原版(sdsu毕业证书)圣地亚哥州立大学毕业证如何办理
 
E-Invoicing Implementation: A Step-by-Step Guide for Saudi Arabian Companies
E-Invoicing Implementation: A Step-by-Step Guide for Saudi Arabian CompaniesE-Invoicing Implementation: A Step-by-Step Guide for Saudi Arabian Companies
E-Invoicing Implementation: A Step-by-Step Guide for Saudi Arabian Companies
 
Preparing Non - Technical Founders for Engaging a Tech Agency
Preparing Non - Technical Founders for Engaging  a  Tech AgencyPreparing Non - Technical Founders for Engaging  a  Tech Agency
Preparing Non - Technical Founders for Engaging a Tech Agency
 
WWDC 2024 Keynote Review: For CocoaCoders Austin
WWDC 2024 Keynote Review: For CocoaCoders AustinWWDC 2024 Keynote Review: For CocoaCoders Austin
WWDC 2024 Keynote Review: For CocoaCoders Austin
 
Project Management: The Role of Project Dashboards.pdf
Project Management: The Role of Project Dashboards.pdfProject Management: The Role of Project Dashboards.pdf
Project Management: The Role of Project Dashboards.pdf
 
Baha Majid WCA4Z IBM Z Customer Council Boston June 2024.pdf
Baha Majid WCA4Z IBM Z Customer Council Boston June 2024.pdfBaha Majid WCA4Z IBM Z Customer Council Boston June 2024.pdf
Baha Majid WCA4Z IBM Z Customer Council Boston June 2024.pdf
 
Unlock the Secrets to Effortless Video Creation with Invideo: Your Ultimate G...
Unlock the Secrets to Effortless Video Creation with Invideo: Your Ultimate G...Unlock the Secrets to Effortless Video Creation with Invideo: Your Ultimate G...
Unlock the Secrets to Effortless Video Creation with Invideo: Your Ultimate G...
 
如何办理(hull学位证书)英国赫尔大学毕业证硕士文凭原版一模一样
如何办理(hull学位证书)英国赫尔大学毕业证硕士文凭原版一模一样如何办理(hull学位证书)英国赫尔大学毕业证硕士文凭原版一模一样
如何办理(hull学位证书)英国赫尔大学毕业证硕士文凭原版一模一样
 
Why Apache Kafka Clusters Are Like Galaxies (And Other Cosmic Kafka Quandarie...
Why Apache Kafka Clusters Are Like Galaxies (And Other Cosmic Kafka Quandarie...Why Apache Kafka Clusters Are Like Galaxies (And Other Cosmic Kafka Quandarie...
Why Apache Kafka Clusters Are Like Galaxies (And Other Cosmic Kafka Quandarie...
 
Migration From CH 1.0 to CH 2.0 and Mule 4.6 & Java 17 Upgrade.pptx
Migration From CH 1.0 to CH 2.0 and  Mule 4.6 & Java 17 Upgrade.pptxMigration From CH 1.0 to CH 2.0 and  Mule 4.6 & Java 17 Upgrade.pptx
Migration From CH 1.0 to CH 2.0 and Mule 4.6 & Java 17 Upgrade.pptx
 
Mobile App Development Company In Noida | Drona Infotech
Mobile App Development Company In Noida | Drona InfotechMobile App Development Company In Noida | Drona Infotech
Mobile App Development Company In Noida | Drona Infotech
 
ALGIT - Assembly Line for Green IT - Numbers, Data, Facts
ALGIT - Assembly Line for Green IT - Numbers, Data, FactsALGIT - Assembly Line for Green IT - Numbers, Data, Facts
ALGIT - Assembly Line for Green IT - Numbers, Data, Facts
 
ppt on the brain chip neuralink.pptx
ppt  on   the brain  chip neuralink.pptxppt  on   the brain  chip neuralink.pptx
ppt on the brain chip neuralink.pptx
 
Safelyio Toolbox Talk Softwate & App (How To Digitize Safety Meetings)
Safelyio Toolbox Talk Softwate & App (How To Digitize Safety Meetings)Safelyio Toolbox Talk Softwate & App (How To Digitize Safety Meetings)
Safelyio Toolbox Talk Softwate & App (How To Digitize Safety Meetings)
 
Everything You Need to Know About X-Sign: The eSign Functionality of XfilesPr...
Everything You Need to Know About X-Sign: The eSign Functionality of XfilesPr...Everything You Need to Know About X-Sign: The eSign Functionality of XfilesPr...
Everything You Need to Know About X-Sign: The eSign Functionality of XfilesPr...
 
ACE - Team 24 Wrapup event at ahmedabad.
ACE - Team 24 Wrapup event at ahmedabad.ACE - Team 24 Wrapup event at ahmedabad.
ACE - Team 24 Wrapup event at ahmedabad.
 
Unveiling the Advantages of Agile Software Development.pdf
Unveiling the Advantages of Agile Software Development.pdfUnveiling the Advantages of Agile Software Development.pdf
Unveiling the Advantages of Agile Software Development.pdf
 
INTRODUCTION TO AI CLASSICAL THEORY TARGETED EXAMPLES
INTRODUCTION TO AI CLASSICAL THEORY TARGETED EXAMPLESINTRODUCTION TO AI CLASSICAL THEORY TARGETED EXAMPLES
INTRODUCTION TO AI CLASSICAL THEORY TARGETED EXAMPLES
 
Microservice Teams - How the cloud changes the way we work
Microservice Teams - How the cloud changes the way we workMicroservice Teams - How the cloud changes the way we work
Microservice Teams - How the cloud changes the way we work
 
UI5con 2024 - Boost Your Development Experience with UI5 Tooling Extensions
UI5con 2024 - Boost Your Development Experience with UI5 Tooling ExtensionsUI5con 2024 - Boost Your Development Experience with UI5 Tooling Extensions
UI5con 2024 - Boost Your Development Experience with UI5 Tooling Extensions
 

Test Selection with Moose In Industry - Impact of Granularity

  • 1. 25 – August – 16 Test Selection with Moose In Industry: Impact of Granularity Vincent Blondeau
  • 2. Vincent Blondeau | 25 - August - 16 | 2 Context ▶ Industrial PhD in a major international IT company –  +7 000 employees –  17 countries –  Problems from the field
  • 3. Vincent Blondeau | 25 - August - 16 | 3 Test Case Selection After a Change ? Test All Takes a long time (3 hours) Test Nothing Not a solution
  • 4. Vincent Blondeau | 25 - August - 16 | 4 Test Case Selection After a Change ▶ Question –  Does test selection modify developers’ habits and enhance software quality? ▶ Goal –  Save time and improve quality –  Select tests to relaunch after a change in the source code –  Any kind of test: End to end, performance… ▶ Comparison of approaches on real changes –  Call graph analysis by static approach –  Comparison with dynamic approach
  • 5. Vincent Blondeau | 25 - August - 16 | 5 Two approaches ▶ Static Approach –  Create a model of the system with Moose –  Navigate the call graph from a changed source code to find the tests Properties –  Allows to find multiple tracks to the changed source –  No test execution
  • 6. Vincent Blondeau | 25 - August - 16 | 6 Two approaches ▶ Dynamic Approach –  Execute the tests •  Map the tests to the covered code –  Relaunch the tests related to changed source code Properties –  Dependent to the test data –  The tests have to be executed
  • 7. Vincent Blondeau | 25 - August - 16 | 7 Issue Classification ▶ Problems in test selection approaches arise when there is a break in the dependency graph representing the system. –  Third-party breaks –  Multi-program breaks –  Dynamic breaks –  Polymorphism breaks
  • 8. Vincent Blondeau | 25 - August - 16 | 8 Experiment ▶ Hypothesis: Dynamic approach is the oracle –  With some flaws: •  Does not work on failing or in error tests •  Requires time to be performed ▶ Approach –  Compare influence of real source code changes –  Simulate code change on several existing projects
  • 9. Vincent Blondeau | 25 - August - 16 | 9 Experiment ▶ Consider real commits by mining repositories –  Weight each covered method by the number of commits –  Group covered methods in commits •  Considered real method commit groups
  • 10. Vincent Blondeau | 25 - August - 16 | 10 Projects: Metrics Metric P1 P2 P3 KLOC Core 447 716 302 # Green Tests 5 323 168 3 035 # Total Methods 9 808 56 661 45 671 # Methods Covered 4720 (48%) 3 261 (6%) 8 143 (18%) #Commits 2 217 467 2 115 Avg Methods/Commit 24 129 37 Avg Files/Commit 7 18 17
  • 11. Vincent Blondeau | 25 - August - 16 | 11 Metrics ▶ Number of selected tests –  Ratio of the total test suite to relaunch ▶ Precision –  How many selected tests are relevant? ▶ Recall –  How many relevant tests are selected?
  • 12. Vincent Blondeau | 25 - August - 16 | 12 #Selected Tests Precision Recall 1 Meth. Weig. 1 Meth. Weig. 1 Meth. Weig. P1 3% 3% 43% 42% 91% 92% P2 0.8% 1% 61% 64% 59% 62% P3 2% 2% 34% 41% 39%33% Weighting of methods with the number of commits
  • 13. Vincent Blondeau | 25 - August - 16 | 13 #Selected Tests Precision Recall 1 Meth. Commit 1 Meth. Commit 1 Meth. Commit P1 3% 4% 43% 55% 91% 81% P2 0.8% 3% 61% 64% 45% 45% P3 2% 6% 49% 41% 56% 34% Methods grouped in commits
  • 14. Vincent Blondeau | 25 - August - 16 | 14 Conclusion ▶ Considering commits instead of individual methods tends to worsen the results ▶ Impact on projects is different ▶ Low ratio of selected tests, so still acceptable Future steps ▶ Better understand how tests are used by developers ▶ Provide a tool for developers to select tests
  • 15. Worldline is a registered trademark of Atos Worldline SAS. June 2013 © 2013 Atos. Confidential information owned by Atos Worldline, to be used by the recipient only. This document, or any part of it, may not be reproduced, copied, circulated and/or distributed nor quoted without prior written approval from Atos Worldline. 25-August-16 Thanks!