SlideShare a Scribd company logo

On the value of Sampling and Pruning for SBSE

Oral Prelim Exam slides (for publication). Thesis statement: for the optimization of SE planning and replanning tasks, given appropriate separation operators, then oversampling and pruning is better than mutation based evolutionary approaches.

1 of 53
Download to read offline
On the Value of Sampling and Pruning
for Search-Based Software Engineering
Jianfeng Chen (jchen37@ncsu.edu)
April 20 2018
1
How to better support SE planning + re-planning?
Plan
(what to do)
Re-plan
(what to react to new
circumstance)
What feature to
include in project
What feature to
include in vi+1
Assign software to
cloud env. How?
Adjust to cloud env.
changes. How?
What to test
first?
What to test
next?
2
Problem: planning & re-planning can be very slow.
Running time SLOW
[Zhang’17] Yuanyuan Zhang, Mark Harman, and A Mansouri. The sbse repository: A repository and analysis of authors and research articles on search based software engineering. CREST Centre, UCL
3
Thesis Statement
For the optimization of SE planning and re-planning tasks,
● given appropriate separation operators1
,
● then OverSampling and Pruning1
(OSAP) is better
● than the mutation based EVOLutionary1
(EVOL) approach
● (where “better” is measured in terms of runtimes, number of
evaluations, and value of final result).
1
to be defined, later in this talk
4
Roadmap
Introduction
EVOL
GALE
OSAP
├─ TopDown Bi-clustering
├─ Encoding Knowledge
└─ Random Anchors
Roadmap
● What is Search-based SE
● EVOL: Evolutionary algorithms
○ GALE: A geometric learner
● OSAP: Oversampling-and-pruning via Separation Operators
5
Roadmap
Introduction
EVOL
GALE
OSAP
├─ TopDown Bi-clustering
├─ Encoding Knowledge
└─ Random Anchors
Publications & tools in this PhD program
FINAL THESISTHIS TALK
[CLOUD18 Chen et al.] (Accept rate: 15%)
RIOT: workflow scheduling tool
[TSE18 Chen et al.]
Sampling as a baseline for SBSE
[IST17 Chen et al.]
Beyond EA for SBSE
[SSBSE16 Nair et al.]
Accidental exploration for SBSE
Publications Tools
6

Recommended

Design Pattern of HBase Configuration
Design Pattern of HBase ConfigurationDesign Pattern of HBase Configuration
Design Pattern of HBase ConfigurationDan Han
 
Lec7 deeprlbootcamp-svg+scg
Lec7 deeprlbootcamp-svg+scgLec7 deeprlbootcamp-svg+scg
Lec7 deeprlbootcamp-svg+scgRonald Teo
 
Mining and Managing Large-scale Linked Open Data
Mining and Managing Large-scale Linked Open DataMining and Managing Large-scale Linked Open Data
Mining and Managing Large-scale Linked Open DataMOVING Project
 
Heaven: Supporting Systematic Comparative Research of RDF Stream Processing E...
Heaven: Supporting Systematic Comparative Research of RDF Stream Processing E...Heaven: Supporting Systematic Comparative Research of RDF Stream Processing E...
Heaven: Supporting Systematic Comparative Research of RDF Stream Processing E...Riccardo Tommasini
 
Tall and Skinny QRs in MapReduce
Tall and Skinny QRs in MapReduceTall and Skinny QRs in MapReduce
Tall and Skinny QRs in MapReduceDavid Gleich
 
Architecting R into Storm Application Development Process
Architecting R into Storm Application Development ProcessArchitecting R into Storm Application Development Process
Architecting R into Storm Application Development ProcessDataWorks Summit
 
Evaluation of Caching Strategies Based on Access Statistics on Past Requests
Evaluation of Caching Strategies Based on Access Statistics on Past RequestsEvaluation of Caching Strategies Based on Access Statistics on Past Requests
Evaluation of Caching Strategies Based on Access Statistics on Past RequestsSmartenIT
 

More Related Content

What's hot

SchemEX - Creating the Yellow Pages for the Linked Open Data Cloud
SchemEX - Creating the Yellow Pages for the Linked Open Data CloudSchemEX - Creating the Yellow Pages for the Linked Open Data Cloud
SchemEX - Creating the Yellow Pages for the Linked Open Data CloudAnsgar Scherp
 
Contributions to the Efficient Use of General Purpose Coprocessors: KDE as Ca...
Contributions to the Efficient Use of General Purpose Coprocessors: KDE as Ca...Contributions to the Efficient Use of General Purpose Coprocessors: KDE as Ca...
Contributions to the Efficient Use of General Purpose Coprocessors: KDE as Ca...Unai Lopez-Novoa
 
How might machine learning help advance solar PV research?
How might machine learning help advance solar PV research?How might machine learning help advance solar PV research?
How might machine learning help advance solar PV research?Anubhav Jain
 
Productivity and Performance: An Exploration of Parallel H5py on HPC
Productivity and Performance: An Exploration of Parallel H5py on HPCProductivity and Performance: An Exploration of Parallel H5py on HPC
Productivity and Performance: An Exploration of Parallel H5py on HPCJialin Liu
 
A TRAINING METHOD USING
 DNN-GUIDED LAYERWISE PRETRAINING
 FOR DEEP GAUSSIAN ...
A TRAINING METHOD USING
 DNN-GUIDED LAYERWISE PRETRAINING
 FOR DEEP GAUSSIAN ...A TRAINING METHOD USING
 DNN-GUIDED LAYERWISE PRETRAINING
 FOR DEEP GAUSSIAN ...
A TRAINING METHOD USING
 DNN-GUIDED LAYERWISE PRETRAINING
 FOR DEEP GAUSSIAN ...Tomoki Koriyama
 

What's hot (9)

SchemEX - Creating the Yellow Pages for the Linked Open Data Cloud
SchemEX - Creating the Yellow Pages for the Linked Open Data CloudSchemEX - Creating the Yellow Pages for the Linked Open Data Cloud
SchemEX - Creating the Yellow Pages for the Linked Open Data Cloud
 
Contributions to the Efficient Use of General Purpose Coprocessors: KDE as Ca...
Contributions to the Efficient Use of General Purpose Coprocessors: KDE as Ca...Contributions to the Efficient Use of General Purpose Coprocessors: KDE as Ca...
Contributions to the Efficient Use of General Purpose Coprocessors: KDE as Ca...
 
How might machine learning help advance solar PV research?
How might machine learning help advance solar PV research?How might machine learning help advance solar PV research?
How might machine learning help advance solar PV research?
 
Productivity and Performance: An Exploration of Parallel H5py on HPC
Productivity and Performance: An Exploration of Parallel H5py on HPCProductivity and Performance: An Exploration of Parallel H5py on HPC
Productivity and Performance: An Exploration of Parallel H5py on HPC
 
Mask R-CNN
Mask R-CNNMask R-CNN
Mask R-CNN
 
A TRAINING METHOD USING
 DNN-GUIDED LAYERWISE PRETRAINING
 FOR DEEP GAUSSIAN ...
A TRAINING METHOD USING
 DNN-GUIDED LAYERWISE PRETRAINING
 FOR DEEP GAUSSIAN ...A TRAINING METHOD USING
 DNN-GUIDED LAYERWISE PRETRAINING
 FOR DEEP GAUSSIAN ...
A TRAINING METHOD USING
 DNN-GUIDED LAYERWISE PRETRAINING
 FOR DEEP GAUSSIAN ...
 
GDRR Opening Workshop - Bayesian Inference for Common Cause Failure Rate Base...
GDRR Opening Workshop - Bayesian Inference for Common Cause Failure Rate Base...GDRR Opening Workshop - Bayesian Inference for Common Cause Failure Rate Base...
GDRR Opening Workshop - Bayesian Inference for Common Cause Failure Rate Base...
 
Pycon9 dibernado
Pycon9 dibernadoPycon9 dibernado
Pycon9 dibernado
 
The Origin of Grad-CAM
The Origin of Grad-CAMThe Origin of Grad-CAM
The Origin of Grad-CAM
 

Similar to On the value of Sampling and Pruning for SBSE

Software Defect Prediction on Unlabeled Datasets
Software Defect Prediction on Unlabeled DatasetsSoftware Defect Prediction on Unlabeled Datasets
Software Defect Prediction on Unlabeled DatasetsSung Kim
 
GALE: Geometric active learning for Search-Based Software Engineering
GALE: Geometric active learning for Search-Based Software EngineeringGALE: Geometric active learning for Search-Based Software Engineering
GALE: Geometric active learning for Search-Based Software EngineeringCS, NcState
 
Compressing Graphs and Indexes with Recursive Graph Bisection
Compressing Graphs and Indexes with Recursive Graph Bisection Compressing Graphs and Indexes with Recursive Graph Bisection
Compressing Graphs and Indexes with Recursive Graph Bisection aftab alam
 
IA3_presentation.pptx
IA3_presentation.pptxIA3_presentation.pptx
IA3_presentation.pptxKtonNguyn2
 
Presentation
PresentationPresentation
Presentationbutest
 
Large-scale Recommendation Systems on Just a PC
Large-scale Recommendation Systems on Just a PCLarge-scale Recommendation Systems on Just a PC
Large-scale Recommendation Systems on Just a PCAapo Kyrölä
 
Chatzikonstantinou c ai-se2013_
Chatzikonstantinou c ai-se2013_Chatzikonstantinou c ai-se2013_
Chatzikonstantinou c ai-se2013_caise2013vlc
 
Dynamically Optimizing Queries over Large Scale Data Platforms
Dynamically Optimizing Queries over Large Scale Data PlatformsDynamically Optimizing Queries over Large Scale Data Platforms
Dynamically Optimizing Queries over Large Scale Data PlatformsINRIA-OAK
 
MMDS 2014: Myria (and Scalable Graph Clustering with RelaxMap)
MMDS 2014: Myria (and Scalable Graph Clustering with RelaxMap)MMDS 2014: Myria (and Scalable Graph Clustering with RelaxMap)
MMDS 2014: Myria (and Scalable Graph Clustering with RelaxMap)University of Washington
 
Transfer defect learning
Transfer defect learningTransfer defect learning
Transfer defect learningSung Kim
 
DYNAMIC SLICING OF ASPECT-ORIENTED PROGRAMS
DYNAMIC SLICING OF ASPECT-ORIENTED PROGRAMSDYNAMIC SLICING OF ASPECT-ORIENTED PROGRAMS
DYNAMIC SLICING OF ASPECT-ORIENTED PROGRAMSPraveen Penumathsa
 
DYNAMIC SLICING OF ASPECT-ORIENTED PROGRAMS
DYNAMIC SLICING OF ASPECT-ORIENTED PROGRAMSDYNAMIC SLICING OF ASPECT-ORIENTED PROGRAMS
DYNAMIC SLICING OF ASPECT-ORIENTED PROGRAMSPraveen Penumathsa
 
OpenPOWER Webinar from University of Delaware - Title :OpenMP (offloading) o...
OpenPOWER Webinar from University of Delaware  - Title :OpenMP (offloading) o...OpenPOWER Webinar from University of Delaware  - Title :OpenMP (offloading) o...
OpenPOWER Webinar from University of Delaware - Title :OpenMP (offloading) o...Ganesan Narayanasamy
 
On the value of sampling and pruning for search-based software engineering
On the value of sampling and pruning for search-based software engineeringOn the value of sampling and pruning for search-based software engineering
On the value of sampling and pruning for search-based software engineeringJianfeng Chen
 
Automated Machine Learning via Sequential Uniform Designs
Automated Machine Learning via Sequential Uniform DesignsAutomated Machine Learning via Sequential Uniform Designs
Automated Machine Learning via Sequential Uniform DesignsAijun Zhang
 
Log Analytics in Datacenter with Apache Spark and Machine Learning
Log Analytics in Datacenter with Apache Spark and Machine LearningLog Analytics in Datacenter with Apache Spark and Machine Learning
Log Analytics in Datacenter with Apache Spark and Machine LearningPiotr Tylenda
 
Log Analytics in Datacenter with Apache Spark and Machine Learning
Log Analytics in Datacenter with Apache Spark and Machine LearningLog Analytics in Datacenter with Apache Spark and Machine Learning
Log Analytics in Datacenter with Apache Spark and Machine LearningAgnieszka Potulska
 

Similar to On the value of Sampling and Pruning for SBSE (20)

Software Defect Prediction on Unlabeled Datasets
Software Defect Prediction on Unlabeled DatasetsSoftware Defect Prediction on Unlabeled Datasets
Software Defect Prediction on Unlabeled Datasets
 
GALE: Geometric active learning for Search-Based Software Engineering
GALE: Geometric active learning for Search-Based Software EngineeringGALE: Geometric active learning for Search-Based Software Engineering
GALE: Geometric active learning for Search-Based Software Engineering
 
Compressing Graphs and Indexes with Recursive Graph Bisection
Compressing Graphs and Indexes with Recursive Graph Bisection Compressing Graphs and Indexes with Recursive Graph Bisection
Compressing Graphs and Indexes with Recursive Graph Bisection
 
IA3_presentation.pptx
IA3_presentation.pptxIA3_presentation.pptx
IA3_presentation.pptx
 
Real Time Geodemographics
Real Time GeodemographicsReal Time Geodemographics
Real Time Geodemographics
 
Presentation
PresentationPresentation
Presentation
 
Large-scale Recommendation Systems on Just a PC
Large-scale Recommendation Systems on Just a PCLarge-scale Recommendation Systems on Just a PC
Large-scale Recommendation Systems on Just a PC
 
Chatzikonstantinou c ai-se2013_
Chatzikonstantinou c ai-se2013_Chatzikonstantinou c ai-se2013_
Chatzikonstantinou c ai-se2013_
 
Dynamically Optimizing Queries over Large Scale Data Platforms
Dynamically Optimizing Queries over Large Scale Data PlatformsDynamically Optimizing Queries over Large Scale Data Platforms
Dynamically Optimizing Queries over Large Scale Data Platforms
 
MMDS 2014: Myria (and Scalable Graph Clustering with RelaxMap)
MMDS 2014: Myria (and Scalable Graph Clustering with RelaxMap)MMDS 2014: Myria (and Scalable Graph Clustering with RelaxMap)
MMDS 2014: Myria (and Scalable Graph Clustering with RelaxMap)
 
Transfer defect learning
Transfer defect learningTransfer defect learning
Transfer defect learning
 
DYNAMIC SLICING OF ASPECT-ORIENTED PROGRAMS
DYNAMIC SLICING OF ASPECT-ORIENTED PROGRAMSDYNAMIC SLICING OF ASPECT-ORIENTED PROGRAMS
DYNAMIC SLICING OF ASPECT-ORIENTED PROGRAMS
 
DYNAMIC SLICING OF ASPECT-ORIENTED PROGRAMS
DYNAMIC SLICING OF ASPECT-ORIENTED PROGRAMSDYNAMIC SLICING OF ASPECT-ORIENTED PROGRAMS
DYNAMIC SLICING OF ASPECT-ORIENTED PROGRAMS
 
OpenPOWER Webinar from University of Delaware - Title :OpenMP (offloading) o...
OpenPOWER Webinar from University of Delaware  - Title :OpenMP (offloading) o...OpenPOWER Webinar from University of Delaware  - Title :OpenMP (offloading) o...
OpenPOWER Webinar from University of Delaware - Title :OpenMP (offloading) o...
 
On the value of sampling and pruning for search-based software engineering
On the value of sampling and pruning for search-based software engineeringOn the value of sampling and pruning for search-based software engineering
On the value of sampling and pruning for search-based software engineering
 
Automated Machine Learning via Sequential Uniform Designs
Automated Machine Learning via Sequential Uniform DesignsAutomated Machine Learning via Sequential Uniform Designs
Automated Machine Learning via Sequential Uniform Designs
 
STRICT-SANER2017
STRICT-SANER2017STRICT-SANER2017
STRICT-SANER2017
 
Fulltext
FulltextFulltext
Fulltext
 
Log Analytics in Datacenter with Apache Spark and Machine Learning
Log Analytics in Datacenter with Apache Spark and Machine LearningLog Analytics in Datacenter with Apache Spark and Machine Learning
Log Analytics in Datacenter with Apache Spark and Machine Learning
 
Log Analytics in Datacenter with Apache Spark and Machine Learning
Log Analytics in Datacenter with Apache Spark and Machine LearningLog Analytics in Datacenter with Apache Spark and Machine Learning
Log Analytics in Datacenter with Apache Spark and Machine Learning
 

Recently uploaded

biofilm fouling of the membrane present in aquaculture
biofilm fouling of the membrane present in aquaculturebiofilm fouling of the membrane present in aquaculture
biofilm fouling of the membrane present in aquacultureVINETUBE2
 
my goal is place in mnc's companies and got good salary
my goal is place in mnc's companies and got good salarymy goal is place in mnc's companies and got good salary
my goal is place in mnc's companies and got good salarymonoarul2004
 
GDSC solution challenge Android ppt.pptx
GDSC solution challenge Android ppt.pptxGDSC solution challenge Android ppt.pptx
GDSC solution challenge Android ppt.pptxAnandMenon54
 
Student Challange as Google Developers at NKOCET
Student Challange as Google Developers at NKOCETStudent Challange as Google Developers at NKOCET
Student Challange as Google Developers at NKOCETGDSCNKOCET
 
Deep Learning For Computer Vision- Day 3 Study Jams GDSC Unsri.pptx
Deep Learning For Computer Vision- Day 3 Study Jams GDSC Unsri.pptxDeep Learning For Computer Vision- Day 3 Study Jams GDSC Unsri.pptx
Deep Learning For Computer Vision- Day 3 Study Jams GDSC Unsri.pptxpmgdscunsri
 
Eversendai - HSE Performance Management Systems-R1.pptx
Eversendai - HSE Performance Management Systems-R1.pptxEversendai - HSE Performance Management Systems-R1.pptx
Eversendai - HSE Performance Management Systems-R1.pptxADILRASHID54
 
Pointers and Array, pointer and String.pptx
Pointers and Array, pointer and String.pptxPointers and Array, pointer and String.pptx
Pointers and Array, pointer and String.pptxAnanthi Palanisamy
 
Nexus - Final Day 12th February 2024.pptx
Nexus - Final Day 12th February 2024.pptxNexus - Final Day 12th February 2024.pptx
Nexus - Final Day 12th February 2024.pptxRohanAgarwal340656
 
Laser And its Application's-Engineering Physics
Laser And its Application's-Engineering PhysicsLaser And its Application's-Engineering Physics
Laser And its Application's-Engineering Physicspurvanikam3
 
ROBOT PERCEPTION FOR AGRICULTURE AND GOOD PRODUCTION1.1.pdf
ROBOT PERCEPTION FOR AGRICULTURE AND GOOD PRODUCTION1.1.pdfROBOT PERCEPTION FOR AGRICULTURE AND GOOD PRODUCTION1.1.pdf
ROBOT PERCEPTION FOR AGRICULTURE AND GOOD PRODUCTION1.1.pdfRudraPratapSingh871925
 
Critical Literature Review Final -MW.pdf
Critical Literature Review Final -MW.pdfCritical Literature Review Final -MW.pdf
Critical Literature Review Final -MW.pdfMollyWinterbottom
 
self introduction sri balaji
self introduction sri balajiself introduction sri balaji
self introduction sri balajiSriBalaji891607
 
MAXIMUM POWER POINT TRACKING ALGORITHMS APPLIED TO WIND-SOLAR HYBRID SYSTEM
MAXIMUM POWER POINT TRACKING ALGORITHMS APPLIED TO WIND-SOLAR HYBRID SYSTEMMAXIMUM POWER POINT TRACKING ALGORITHMS APPLIED TO WIND-SOLAR HYBRID SYSTEM
MAXIMUM POWER POINT TRACKING ALGORITHMS APPLIED TO WIND-SOLAR HYBRID SYSTEMArunkumar Tulasi
 
nervous system ppt pptx anatomy system of nerves
nervous system ppt pptx anatomy system of nervesnervous system ppt pptx anatomy system of nerves
nervous system ppt pptx anatomy system of nervesPhebeLois1
 
Metrology Measurements and All units PPT
Metrology Measurements and  All units PPTMetrology Measurements and  All units PPT
Metrology Measurements and All units PPTdinesh babu
 
Architectural Preservation - Heritage, focused on Saudi Arabia
Architectural Preservation - Heritage, focused on Saudi ArabiaArchitectural Preservation - Heritage, focused on Saudi Arabia
Architectural Preservation - Heritage, focused on Saudi ArabiaIgnacio J. Palma, Arch PhD.
 
Sample Case Study of industry 4.0 and its Outcome
Sample Case Study of industry 4.0 and its OutcomeSample Case Study of industry 4.0 and its Outcome
Sample Case Study of industry 4.0 and its OutcomeHarshith A S
 
Deluck Technical Works Company Profile.pdf
Deluck Technical Works Company Profile.pdfDeluck Technical Works Company Profile.pdf
Deluck Technical Works Company Profile.pdfartpoa9
 
Introduction to Machine Learning Unit-1 Notes for II-II Mechanical Engineerin...
Introduction to Machine Learning Unit-1 Notes for II-II Mechanical Engineerin...Introduction to Machine Learning Unit-1 Notes for II-II Mechanical Engineerin...
Introduction to Machine Learning Unit-1 Notes for II-II Mechanical Engineerin...C Sai Kiran
 
Microstrip Bandpass Filter Design using EDA Tolol such as keysight ADS and An...
Microstrip Bandpass Filter Design using EDA Tolol such as keysight ADS and An...Microstrip Bandpass Filter Design using EDA Tolol such as keysight ADS and An...
Microstrip Bandpass Filter Design using EDA Tolol such as keysight ADS and An...GauravBhartie
 

Recently uploaded (20)

biofilm fouling of the membrane present in aquaculture
biofilm fouling of the membrane present in aquaculturebiofilm fouling of the membrane present in aquaculture
biofilm fouling of the membrane present in aquaculture
 
my goal is place in mnc's companies and got good salary
my goal is place in mnc's companies and got good salarymy goal is place in mnc's companies and got good salary
my goal is place in mnc's companies and got good salary
 
GDSC solution challenge Android ppt.pptx
GDSC solution challenge Android ppt.pptxGDSC solution challenge Android ppt.pptx
GDSC solution challenge Android ppt.pptx
 
Student Challange as Google Developers at NKOCET
Student Challange as Google Developers at NKOCETStudent Challange as Google Developers at NKOCET
Student Challange as Google Developers at NKOCET
 
Deep Learning For Computer Vision- Day 3 Study Jams GDSC Unsri.pptx
Deep Learning For Computer Vision- Day 3 Study Jams GDSC Unsri.pptxDeep Learning For Computer Vision- Day 3 Study Jams GDSC Unsri.pptx
Deep Learning For Computer Vision- Day 3 Study Jams GDSC Unsri.pptx
 
Eversendai - HSE Performance Management Systems-R1.pptx
Eversendai - HSE Performance Management Systems-R1.pptxEversendai - HSE Performance Management Systems-R1.pptx
Eversendai - HSE Performance Management Systems-R1.pptx
 
Pointers and Array, pointer and String.pptx
Pointers and Array, pointer and String.pptxPointers and Array, pointer and String.pptx
Pointers and Array, pointer and String.pptx
 
Nexus - Final Day 12th February 2024.pptx
Nexus - Final Day 12th February 2024.pptxNexus - Final Day 12th February 2024.pptx
Nexus - Final Day 12th February 2024.pptx
 
Laser And its Application's-Engineering Physics
Laser And its Application's-Engineering PhysicsLaser And its Application's-Engineering Physics
Laser And its Application's-Engineering Physics
 
ROBOT PERCEPTION FOR AGRICULTURE AND GOOD PRODUCTION1.1.pdf
ROBOT PERCEPTION FOR AGRICULTURE AND GOOD PRODUCTION1.1.pdfROBOT PERCEPTION FOR AGRICULTURE AND GOOD PRODUCTION1.1.pdf
ROBOT PERCEPTION FOR AGRICULTURE AND GOOD PRODUCTION1.1.pdf
 
Critical Literature Review Final -MW.pdf
Critical Literature Review Final -MW.pdfCritical Literature Review Final -MW.pdf
Critical Literature Review Final -MW.pdf
 
self introduction sri balaji
self introduction sri balajiself introduction sri balaji
self introduction sri balaji
 
MAXIMUM POWER POINT TRACKING ALGORITHMS APPLIED TO WIND-SOLAR HYBRID SYSTEM
MAXIMUM POWER POINT TRACKING ALGORITHMS APPLIED TO WIND-SOLAR HYBRID SYSTEMMAXIMUM POWER POINT TRACKING ALGORITHMS APPLIED TO WIND-SOLAR HYBRID SYSTEM
MAXIMUM POWER POINT TRACKING ALGORITHMS APPLIED TO WIND-SOLAR HYBRID SYSTEM
 
nervous system ppt pptx anatomy system of nerves
nervous system ppt pptx anatomy system of nervesnervous system ppt pptx anatomy system of nerves
nervous system ppt pptx anatomy system of nerves
 
Metrology Measurements and All units PPT
Metrology Measurements and  All units PPTMetrology Measurements and  All units PPT
Metrology Measurements and All units PPT
 
Architectural Preservation - Heritage, focused on Saudi Arabia
Architectural Preservation - Heritage, focused on Saudi ArabiaArchitectural Preservation - Heritage, focused on Saudi Arabia
Architectural Preservation - Heritage, focused on Saudi Arabia
 
Sample Case Study of industry 4.0 and its Outcome
Sample Case Study of industry 4.0 and its OutcomeSample Case Study of industry 4.0 and its Outcome
Sample Case Study of industry 4.0 and its Outcome
 
Deluck Technical Works Company Profile.pdf
Deluck Technical Works Company Profile.pdfDeluck Technical Works Company Profile.pdf
Deluck Technical Works Company Profile.pdf
 
Introduction to Machine Learning Unit-1 Notes for II-II Mechanical Engineerin...
Introduction to Machine Learning Unit-1 Notes for II-II Mechanical Engineerin...Introduction to Machine Learning Unit-1 Notes for II-II Mechanical Engineerin...
Introduction to Machine Learning Unit-1 Notes for II-II Mechanical Engineerin...
 
Microstrip Bandpass Filter Design using EDA Tolol such as keysight ADS and An...
Microstrip Bandpass Filter Design using EDA Tolol such as keysight ADS and An...Microstrip Bandpass Filter Design using EDA Tolol such as keysight ADS and An...
Microstrip Bandpass Filter Design using EDA Tolol such as keysight ADS and An...
 

On the value of Sampling and Pruning for SBSE

  • 1. On the Value of Sampling and Pruning for Search-Based Software Engineering Jianfeng Chen (jchen37@ncsu.edu) April 20 2018 1
  • 2. How to better support SE planning + re-planning? Plan (what to do) Re-plan (what to react to new circumstance) What feature to include in project What feature to include in vi+1 Assign software to cloud env. How? Adjust to cloud env. changes. How? What to test first? What to test next? 2
  • 3. Problem: planning & re-planning can be very slow. Running time SLOW [Zhang’17] Yuanyuan Zhang, Mark Harman, and A Mansouri. The sbse repository: A repository and analysis of authors and research articles on search based software engineering. CREST Centre, UCL 3
  • 4. Thesis Statement For the optimization of SE planning and re-planning tasks, ● given appropriate separation operators1 , ● then OverSampling and Pruning1 (OSAP) is better ● than the mutation based EVOLutionary1 (EVOL) approach ● (where “better” is measured in terms of runtimes, number of evaluations, and value of final result). 1 to be defined, later in this talk 4
  • 5. Roadmap Introduction EVOL GALE OSAP ├─ TopDown Bi-clustering ├─ Encoding Knowledge └─ Random Anchors Roadmap ● What is Search-based SE ● EVOL: Evolutionary algorithms ○ GALE: A geometric learner ● OSAP: Oversampling-and-pruning via Separation Operators 5
  • 6. Roadmap Introduction EVOL GALE OSAP ├─ TopDown Bi-clustering ├─ Encoding Knowledge └─ Random Anchors Publications & tools in this PhD program FINAL THESISTHIS TALK [CLOUD18 Chen et al.] (Accept rate: 15%) RIOT: workflow scheduling tool [TSE18 Chen et al.] Sampling as a baseline for SBSE [IST17 Chen et al.] Beyond EA for SBSE [SSBSE16 Nair et al.] Accidental exploration for SBSE Publications Tools 6
  • 7. Roadmap Introduction EVOL GALE OSAP ├─ TopDown Bi-clustering ├─ Encoding Knowledge └─ Random Anchors Roadmap ● What is Search-based SE ● EVOL: Evolutionary algorithms ○ GALE: A geometric learner ● OSAP: Oversampling-and-pruning via Separation Operators 7
  • 8. SE = making choices in multi (rival) objectives ● Deployments (improving QoS vs. reducing deployment cost) ○ CLOUD: cloud configuration optimization ● Testing (test cost vs. defects detected) ○ Fuzzy testing: less test cases to cover more paths ● SE Planning (trade offs functionality vs. cost) ○ NRP: next release requirements planning ○ SPL: software product lines: product selection Roadmap Introduction EVOL GALE OSAP ├─ TopDown Bi-clustering ├─ Encoding Knowledge └─ Random Anchors 8
  • 9. Res s Tim Cos Search based Software Engineering (SBSE) converts a software engineering problem into a computational search problem, and solves that. Roadmap Introduction EVOL GALE OSAP ├─ TopDown Bi-clustering ├─ Encoding Knowledge └─ Random Anchors Mem A c C U l o Ban d b f(b) f(a) a 9
  • 10. Configuration Space Objective Space Dominance: p dominates q if and only if Consider every objective, p performs no worse than q AND There exist at least one objective, p preforms strictly better than q Roadmap Introduction EVOL GALE OSAP ├─ TopDown Bi-clustering ├─ Encoding Knowledge └─ Random Anchors f(p) f(q)f(x) 10 Pareto frontier Res s Tim co
  • 11. Configuration Space Objective Space Roadmap Introduction EVOL GALE OSAP ├─ TopDown Bi-clustering ├─ Encoding Knowledge └─ Random Anchors f(p) f(q)f(x) Characteristics of SBSE problems ● More than one objective ● Configuration space is huge ● Constrained configurations ● Complex (no easy to assess configurations) 11In SBSE community: the Evolutionary algorithm
  • 12. Roadmap Introduction EVOL GALE OSAP ├─ TopDown Bi-clustering ├─ Encoding Knowledge └─ Random Anchors Roadmap ● What is Search-based SE ● EVOL: Evolutionary algorithms ○ GALE: A geometric learner ● OSAP: Oversampling-and-pruning via Separation Operators 12
  • 13. initial configurations (population) best configurations Treat the problem as black-box Easy to deploy to new problem ~~SLOW~~ ● Airspace operation model verification -- 7 days [Krall’15] ● Test suite generation -- weeks [Yoo’12] ● Software clone evaluation @ pc -- 15 years [Wang’13] Krall, Joseph, Tim Menzies, and Misty Davies. "Learning the task management space of an aircraft approach model." (2014). Yoo, Shin, and Mark Harman. "Regression testing minimization, selection and prioritization: a survey." Software Testing, Verification and Reliability 22.2 (2012): 67-120. Wang, Tiantian, et al. "Searching for better configurations: a rigorous approach to clone evaluation." Proceedings of the 2013 9th Joint Meeting on Foundations of Software Engineering. ACM, 2013. Roadmap Introduction EVOL GALE OSAP ├─ TopDown Bi-clustering ├─ Encoding Knowledge └─ Random Anchors Evolutionary algorithm (EVOL) 13
  • 14. Chang, C. K., Jiang, H. Y., Di, Y., Zhu, D., & Ge, Y. (2008). Time-line based model for software project scheduling with genetic algorithms. Information and Software Technology, 50(11) Tsai, Chun-Wei, et al. "A hyper-heuristic scheduling algorithm for cloud." IEEE Transactions on Cloud Computing 2.2 (2014): 236-250. Arcuri, Andrea. "Many Independent Objective (MIO) Algorithm for Test Suite Generation." International Symposium on Search Based Software Engineering. Springer, Cham, 2017. Research directions in SBSE 2 Combining EAs E.g. [Tsai’14] A Hyper-heuristic Scheduling Algorithm for cloud GA+SA+ACO+PSO Slow^2 3 Re-design objective functions E.g. [Andrea’17] Many Independent objective algorithm for test suite generation Much complex model. Longer time to evaluate Roadmap Introduction EVOL GALE OSAP ├─ TopDown Bi-clustering ├─ Encoding Knowledge └─ Random Anchors 1 Better configuration encoding E.g. [Chang’11] Time-line based model for software project scheduling with genetic algorithm Expert knowledge; carefully design recombination/mutation 14
  • 15. Roadmap Introduction EVOL GALE OSAP ├─ TopDown Bi-clustering ├─ Encoding Knowledge └─ Random Anchors Current SBSE solutions are too slow ! Why need faster optimizers? (Save $$$, Faster response to model changes) 15
  • 16. Roadmap Introduction EVOL GALE OSAP ├─ TopDown Bi-clustering ├─ Encoding Knowledge └─ Random Anchors Roadmap ● What is Search-based SE ● EVOL: Evolutionary algorithms ○ GALE: A geometric learner ● OSAP: Oversampling-and-pruning via Separation Operators 16
  • 17. Roadmap Introduction EVOL GALE OSAP ├─ TopDown Bi-clustering ├─ Encoding Knowledge └─ Random Anchors [Krall’15] Krall, Joseph, Tim Menzies, and Misty Davies. "Gale: Geometric active learning for search-based software engineering." TSE Configuration Space GALE = Geometric active learner [krall’15] 17 initial configurations (population) best configurations Objective Space
  • 18. Roadmap Introduction EVOL GALE OSAP ├─ TopDown Bi-clustering ├─ Encoding Knowledge └─ Random Anchors [Krall’15] Krall, Joseph, Tim Menzies, and Misty Davies. "Gale: Geometric active learning for search-based software engineering." TSE Configuration Space GALE = Geometric active learner [krall’15] 18 best configurations Objective Space initial configurations (population)
  • 19. Roadmap Introduction EVOL GALE OSAP ├─ TopDown Bi-clustering ├─ Encoding Knowledge └─ Random Anchors [Krall’15] Krall, Joseph, Tim Menzies, and Misty Davies. "Gale: Geometric active learning for search-based software engineering." TSE GALE = Geometric active learner [krall’15] EVOL GALE Population N = 100 N = 100 Recombination ✓ ✓ Mutation ✓ ✓ Evaluation # gen# * N gen# * 2*log(N) O(G·N) -> O(G·logN) 19
  • 20. Roadmap Introduction EVOL GALE OSAP ├─ TopDown Bi-clustering ├─ Encoding Knowledge └─ Random Anchors [Krall’15] Krall, Joseph, Tim Menzies, and Misty Davies. "Gale: Geometric active learning for search-based software engineering." TSE GALE = Geometric active learner [krall’15] 20
  • 21. Roadmap Introduction EVOL GALE OSAP ├─ TopDown Bi-clustering ├─ Encoding Knowledge └─ Random Anchors [Krall’15] Krall, Joseph, Tim Menzies, and Misty Davies. "Gale: Geometric active learning for search-based software engineering." TSE Configuration Space Objective Space The selected configuration region did not swift a lot. Not necessary to explore more generations. Increase population size. [100->10,000] Over-sampling 21
  • 22. Roadmap Introduction EVOL GALE OSAP ├─ TopDown Bi-clustering ├─ Encoding Knowledge └─ Random Anchors OSAP Oversampling and pruning EVOL GALE Over-sampling Population N = 100 N = 100 Recombination ✓ ✓ Mutation ✓ ✓ Evaluation # gen# * N gen# * 2log(N) O(G·N) -> O(G·logN)-> O(logN) N=10,000 ✘ ✘ 2log(N) ... Over-sampling: population is much larger 22
  • 23. Roadmap Introduction EVOL GALE OSAP ├─ TopDown Bi-clustering ├─ Encoding Knowledge └─ Random Anchors Roadmap ● What is Search-based SE ● EVOL: Evolutionary algorithms ○ GALE: A geometric learner ● OSAP: Oversampling-and-pruning via Separation Operators Separation Operators 1 Top-down bi-clustering Algorithm Configuration Space Study Cases 23
  • 24. Roadmap Introduction EVOL GALE OSAP ├─ TopDown Bi-clustering ├─ Encoding Knowledge └─ Random Anchors SWAY = Top-down bi-clustering (R) rand init configuration (W) Furthest to (R) (E)Furthest to (W) Configuration Space 24 “Diameter” of configuration space
  • 25. Roadmap Introduction EVOL GALE OSAP ├─ TopDown Bi-clustering ├─ Encoding Knowledge └─ Random Anchors SWAY = Top-down bi-clustering (W) Furthest to (R) (E)Furthest to (W) Configuration Space 25 “Diameter” of configuration space
  • 26. Roadmap Introduction EVOL GALE OSAP ├─ TopDown Bi-clustering ├─ Encoding Knowledge └─ Random Anchors Separation Operators 1 Top-down bi-clustering Algorithm SWAY Configuration Space Continuous Study Cases XOMO, POM3 26 Chen, Jianfeng, et al. "" Sampling" as a Baseline Optimizer for Search-based Software Engineering." IEEE Transactions on Software Engineering (2018).
  • 27. Roadmap Introduction EVOL GALE OSAP ├─ TopDown Bi-clustering ├─ Encoding Knowledge └─ Random Anchors Assuming: small region of configuration space can lead to the frontier What if Configuration Space Objective Space 27
  • 28. Roadmap Introduction EVOL GALE OSAP ├─ TopDown Bi-clustering ├─ Encoding Knowledge └─ Random Anchors Configuration Space Objective Space Perform the top-down bi-clustering separately 28
  • 29. Roadmap Introduction EVOL GALE OSAP ├─ TopDown Bi-clustering ├─ Encoding Knowledge └─ Random Anchors Configuration Space Objective Space Encoding: represent the model configuration in vectors, combinations, etc. How the model encoded? How can we gather similar configurations ? SWAY2 , Separate via Encoding knowledge 29
  • 30. Software Product Line optimization Objectives Select features to develop such that... ● More features ● Less defects ● Less total cost ● More familiar features Roadmap Introduction EVOL GALE OSAP ├─ TopDown Bi-clustering ├─ Encoding Knowledge └─ Random Anchors 30
  • 31. Software Product Line optimization Configuration (feature model) Roadmap Introduction EVOL GALE OSAP ├─ TopDown Bi-clustering ├─ Encoding Knowledge └─ Random Anchors optionalmandatory Cross tree constraints 31
  • 32. Roadmap Introduction EVOL GALE OSAP ├─ TopDown Bi-clustering ├─ Encoding Knowledge └─ Random Anchors Software Product Line optimization CNF (conjunctive normal forms) Solvable by SAT solvers. Initialization via SAT solver. 32
  • 33. Roadmap Introduction EVOL GALE OSAP ├─ TopDown Bi-clustering ├─ Encoding Knowledge └─ Random Anchors Software Product Line optimization CNF (conjunctive normal forms) Solvable by SAT solvers. Initialization via SAT solver. HIGH DIMENSIONAL HIGHLY CONSTRAINED 33
  • 34. Roadmap Introduction EVOL GALE OSAP ├─ TopDown Bi-clustering ├─ Encoding Knowledge └─ Random Anchors Software Product Line optimization Related work (EVOL) White, Jules, Brian Doughtery, and Douglas C. Schmidt. "Filtered Cartesian Flattening: An Approximation Technique for Optimally Selecting Features while Adhering to Resource Constraints." SPLC (2). 2008. Wu, Zhiqiao, et al. "An optimization model for reuse scenario selection considering reliability and cost in software product line development." International Journal of Information Technology & Decision Making 10.05 (2011): 811-841. Sayyad, Abdel Salam, Tim Menzies, and Hany Ammar. "On the value of user preferences in search-based software engineering: a case study in software product lines." ICSE’13 Sayyad, Abdel Salam, et al. "Scalable product line configuration: A straw to break the camel's back." Automated Software Engineering (ASE), 2013 Henard, Christopher, et al. "Combining multi-objective search and constraint solving for configuring large software product lines." Software Engineering (ICSE), 2015 White’08 Wu’11 Sayyad’13 Henard’15 Single obj Aggregated obj IBEA 34
  • 35. Roadmap Introduction EVOL GALE OSAP ├─ TopDown Bi-clustering ├─ Encoding Knowledge └─ Random Anchors How the model encoded? How can we gather similar configurations ? As scale increases Scale = 4 Configuration Space Objective Space co s de t 35
  • 36. Roadmap Introduction EVOL GALE OSAP ├─ TopDown Bi-clustering ├─ Encoding Knowledge └─ Random Anchors As scale increasesscale Radius ∝ scale Inner circle :: smaller area :: less diverse for simple configurations Outer circle :: larger area :: larger diverse for complex configurations 36
  • 37. Roadmap Introduction EVOL GALE OSAP ├─ TopDown Bi-clustering ├─ Encoding Knowledge └─ Random Anchors Radius ∝ scale Smaller area. Less configurations Larger area. More configurations 37 Configuration Space Objective Space co s de t
  • 38. Roadmap Introduction EVOL GALE OSAP ├─ TopDown Bi-clustering ├─ Encoding Knowledge └─ Random Anchors constraints#. i.e. complexity of the model State-of-the-art EVOL SWAY2 is (orders of magnitude) faster than EVOL. This is important when models become complex 38
  • 39. Roadmap Introduction EVOL GALE OSAP ├─ TopDown Bi-clustering ├─ Encoding Knowledge └─ Random Anchors Wang, Shuai, et al. "A practical guide to select quality indicators for assessing pareto-based search algorithms in search-based software engineering." Software Engineering (ICSE), 2016 IEEE/ACM 38th International Conference on. IEEE, 2016. GS PFS HV Webportal 81 Eshop 506 Fiasco 5228 Freebsd 62138 Linux 343944 Obtained frontiers Pareto front size (PFS) # of obtained frontiers Hyper-volume (HV) Spread (GS) 39
  • 40. Roadmap Introduction EVOL GALE OSAP ├─ TopDown Bi-clustering ├─ Encoding Knowledge └─ Random Anchors SWAY(*) VS. State-Of-The-Art ⬤ Statistically no difference than SATIBEA ⬤ Significantly better than SATIBEA ⬤ Significantly worse than SATIBEA 40 A12 >= 0.6, not the same Arcuri and Briand at ICSE’11 Arcuri, Andrea, and Lionel Briand. "A practical guide for using statistical tests to assess randomized algorithms in software engineering." Software Engineering (ICSE), 2011 33rd International Conference on. IEEE, 2011.
  • 41. Roadmap Introduction EVOL GALE OSAP ├─ TopDown Bi-clustering ├─ Encoding Knowledge └─ Random Anchors W/o encoding knowledge GS PFS HV Webportal 81 ⬤ ⬤ ⬤ Eshop 506 ⬤ ⬤ ⬤ Fiasco 5228 ⬤ ⬤ ⬤ Freebsd 62138 ⬤ ⬤ ⬤ Linux 343944 ⬤ ⬤ ⬤ SWAY(*) VS. State-Of-The-Art ⬤ Statistically no difference than SATIBEA ⬤ Significantly better than SATIBEA ⬤ Significantly worse than SATIBEA With encoding knowledge GS PFS HV Webportal ⬤ ⬤ ⬤ eshop ⬤ ⬤ ⬤ Fiasco ⬤ ⬤ ⬤ freebsd ⬤ ⬤ ⬤ linux ⬤ ⬤ ⬤ 41Arcuri, Andrea, and Lionel Briand. "A practical guide for using statistical tests to assess randomized algorithms in software engineering." Software Engineering (ICSE), 2011 33rd International Conference on. IEEE, 2011.
  • 42. Roadmap Introduction EVOL GALE OSAP ├─ TopDown Bi-clustering ├─ Encoding Knowledge └─ Random Anchors W/o encoding knowledge GS PFS HV Webportal 81 ⬤ ⬤ ⬤ Eshop 506 ⬤ ⬤ ⬤ Fiasco 5228 ⬤ ⬤ ⬤ Freebsd 62138 ⬤ ⬤ ⬤ Linux 343944 ⬤ ⬤ ⬤ SWAY(*) VS. State-Of-The-Art ⬤ Statistically no difference than SATIBEA ⬤ Significantly better than SATIBEA ⬤ Significantly worse than SATIBEA With encoding knowledge GS PFS HV Webportal ⬤ ⬤ ⬤ eshop ⬤ ⬤ ⬤ Fiasco ⬤ ⬤ ⬤ freebsd ⬤ ⬤ ⬤ linux ⬤ ⬤ ⬤ Across all measures, in the majority cases, SWAY2 is better than SATIBEA (EVOL) 42Arcuri, Andrea, and Lionel Briand. "A practical guide for using statistical tests to assess randomized algorithms in software engineering." Software Engineering (ICSE), 2011 33rd International Conference on. IEEE, 2011.
  • 43. Roadmap Introduction EVOL GALE OSAP ├─ TopDown Bi-clustering ├─ Encoding Knowledge └─ Random Anchors Separation Operators 1 Top-down bi-clustering 2 Encoding Knowledge Algorithm SWAY SWAY2 Configuration Space Continuous Binary vector Highly constrained Study Cases XOMO, POM3 SPL 43
  • 44. Roadmap Introduction EVOL GALE OSAP ├─ TopDown Bi-clustering ├─ Encoding Knowledge └─ Random Anchors Configuration Space Objective Space Q: How to find the complete frontier? A: Increase the “resolution” of the separation However, we can’t evaluate too many configurations! 44
  • 45. Roadmap Introduction EVOL GALE OSAP ├─ TopDown Bi-clustering ├─ Encoding Knowledge └─ Random Anchors Select and evaluate a few “representative” configurations -- anchors. # anchors << # init configurations Choices of anchors: ★ 1 = the diagonal ★ 2 = random ★ 3 = 1 + 2 45
  • 46. Roadmap Introduction EVOL GALE OSAP ├─ TopDown Bi-clustering ├─ Encoding Knowledge └─ Random Anchors Select and evaluate a few “representative” configurations -- anchors. Then use the evaluated anchors to guess objectives of the other configurations Surrogate model: replace the origin complex model with a very simple model/formula. Config to guess “c” Nearest anchor N Similar config-> similar objs Furthest anchor F p Q p:Q 46 xY x:Y = p:Q f(c) f(N) f(F) O1
  • 47. Roadmap Introduction EVOL GALE OSAP ├─ TopDown Bi-clustering ├─ Encoding Knowledge └─ Random Anchors Workflow deployments MONTAGE NASA workflow for generating custom images of the sky task workflow Objectives Select proper virtual machines to execute each task so that ... ● end workflow earlier ● less cloud service rental cost Configuration space 47RIOT: Randomized instance types
  • 48. Roadmap Introduction EVOL GALE OSAP ├─ TopDown Bi-clustering ├─ Encoding Knowledge └─ Random Anchors Zhu, Zhaomeng, et al. "Evolutionary multi-objective workflow scheduling in cloud." IEEE Transactions on parallel and distributed Systems 27.5 (2016): 1344-1357. Finish time if we deploy model to aws using median $$$ State-of-the-art method [Zhang’16]. EVOL based 48
  • 49. Roadmap Introduction EVOL GALE OSAP ├─ TopDown Bi-clustering ├─ Encoding Knowledge └─ Random Anchors Zhu, Zhaomeng, et al. "Evolutionary multi-objective workflow scheduling in cloud." IEEE Transactions on parallel and distributed Systems 27.5 (2016): 1344-1357. 49 Montage as tasks # increases Epigenomics Inspiral Cybershake Sipht y=speedup EVOL/RIOT
  • 50. Roadmap Introduction EVOL GALE OSAP ├─ TopDown Bi-clustering ├─ Encoding Knowledge └─ Random Anchors Zhu, Zhaomeng, et al. "Evolutionary multi-objective workflow scheduling in cloud." IEEE Transactions on parallel and distributed Systems 27.5 (2016): 1344-1357. 50 Montage as tasks # increases Epigenomics Inspiral Cybershake Sipht y=speedup EVOL/RIOT RIOT is much faster than state-of-the-art(EVOL)
  • 51. Roadmap Introduction EVOL GALE OSAP ├─ TopDown Bi-clustering ├─ Encoding Knowledge └─ Random Anchors Zhu, Zhaomeng, et al. "Evolutionary multi-objective workflow scheduling in cloud." IEEE Transactions on parallel and distributed Systems 27.5 (2016): 1344-1357. Obtained frontiers Hyper-volume (HV) Spread (GS) Bold blue values RIOT performed as well as or better than state-of-the-art EVOL Across all measures, in the majority cases, statistically, RIOT is better than EVOL. 51
  • 52. Roadmap Introduction EVOL GALE OSAP ├─ TopDown Bi-clustering ├─ Encoding Knowledge └─ Random Anchors Recap ● EVOL: Evolutionary algorithms ● OSAP: Oversampling-and-pruning via Separation Operators Separation Operators 1 Top-down bi-clustering 2 Encoding Knowledge 3 Random Anchors Algorithm SWAY SWAY2 RIOT Configuration Space Continuous Binary vector Highly constrained Enumerates Study Cases XOMO, POM3 SPL Workflow config 52
  • 53. Roadmap Introduction EVOL GALE OSAP ├─ TopDown Bi-clustering ├─ Encoding Knowledge └─ Random Anchors Conclusion For the optimization of SE planning and re-planning tasks, ● given appropriate separation operators, ● then over-sampling+pruning (OSAP) is better ● than the standard mutation+evolutionary (EVOL) approach 53