SlideShare a Scribd company logo
1 of 39
Download to read offline
1
src= tiny.cc/gale15code
slides= tiny.cc/gale15
GALE: Geometric active learning for
Search-Based Software Engineering
Joseph Krall, LoadIQ
Tim Menzies, NC State
Misty Davies, NASA Ames
Sept 5, 2015
Slides: tiny.cc/gale15
Software: tiny.cc/gale15code
10th Joint Meeting of the European Software Engineering Conference and the ACM SIGSOFT
Symposium on the Foundations of Software Engineering: FSE’15
ai4se.net
2
src= tiny.cc/gale15code
slides= tiny.cc/gale15
This talk
• What is search-based SE?
• Why use less CPU for SBSE?
• How to use less CPU
– Refactor the optimizer:
– Add in some data mining
• Experimental results
• Related Work
• Future work
• A challenge question:
– Are we making this too hard?
ai4se.net
4
src= tiny.cc/gale15code
slides= tiny.cc/gale15
This talk
• What is search-based SE?
• Why use less CPU for SBSE?
• How to use less CPU
– Refactor the optimizer:
– Add in some data mining
• Experimental results
• Related Work
• Future work
• A challenge question:
– Are we making this too hard?
ai4se.net
5ai4se.net
src= tiny.cc/gale15code
slides= tiny.cc/gale15
Q: What is Search-based SE?
A: The future
• Ye olde SE
– Manually code up your
understanding a domain
– Struggle to understand
that software
• Search-based-
model-based SE
– Code up domain knowledge
into a model
– Explore that model
– All models are wrong
• But some are useful
6
src= tiny.cc/gale15code
slides= tiny.cc/gale15
SBSE = everything
1. Requirements Menzies, Feather, Bagnall, Mansouri, Zhang
2. Transformation Cooper, Ryan, Schielke, Subramanian, Fatiregun, Williams
3.Effort prediction Aguilar-Ruiz, Burgess, Dolado, Lefley, Shepperd
4. Management Alba, Antoniol, Chicano, Di Pentam Greer, Ruhe
5. Heap allocation Cohen, Kooi, Srisa-an
6. Regression test Li, Yoo, Elbaum, Rothermel, Walcott, Soffa, Kampfhamer
7. SOA Canfora, Di Penta, Esposito, Villani
8. Refactoring Antoniol, Briand, Cinneide, O’Keeffe, Merlo, Seng, Tratt
9. Test Generation Alba, Binkley, Bottaci, Briand, Chicano, Clark, Cohen, Gutjahr, Harrold, Holcombe, Jones,
Korel, Pargass, Reformat, Roper, McMinn, Michael, Sthamer, Tracy, Tonella,Xanthakis, Xiao,
Wegener, Wilkins
10. Maintenance Antoniol, Lutz, Di Penta, Madhavi, Mancoridis, Mitchell, Swift
11. Model checking Alba, Chicano, Godefroid
12. Probing Cohen, Elbaum
13. UIOs Derderian, Guo, Hierons
14. Comprehension Gold, Li, Mahdavi
15. Protocols Alba, Clark, Jacob, Troya
16. Component sel Baker, Skaliotis, Steinhofel, Yoo
17. Agent Oriented Haas, Peysakhov, Sinclair, Shami, Mancoridisai4se.net
7
src= tiny.cc/gale15code
slides= tiny.cc/gale15
SBSE = CPU-intensive
Explosive growth of SBSE papers
ai4se.net
8
src= tiny.cc/gale15code
slides= tiny.cc/gale15
SBSE = CPU-intensive
Evaluates 1000s, 1,000,000s of candidates
Objectives = evaluate(decisions)
Cost = Generations * (Selection + Evaluation * Generation)
G * (O(N2) + E * O(1) * N)
Explosive growth of SBSE papers
ai4se.net
9
src= tiny.cc/gale15code
slides= tiny.cc/gale15
This talk
• What is search-based SE?
• Why use less CPU for SBSE?
• How to use less CPU
– Refactor the optimizer:
– Add in some data mining
• Experimental results
• Related Work
• Future work
• A challenge question:
– Are we making this too hard?
ai4se.net
10ai4se.net
src= tiny.cc/gale15code
slides= tiny.cc/gale15
• Less power
– Less power generation
pollution
– Less barriers to usage
• Less cost
– of hardware of cloud
time
Why seek less CPU?
11ai4se.net
src= tiny.cc/gale15code
slides= tiny.cc/gale15
• Less generation of
candidates
– Less confusion
• Verrappa and Letier:
– “..for industrial
problems, these
algorithms generate
(many) solutions
(makes)
understanding them
and selecting one
among them difficult
and time
consuming”
https://goo.gl/LvsQd
n
Why seek less CPU?
12ai4se.net
src= tiny.cc/gale15code
slides= tiny.cc/gale15
When searching for solutions
“you don’t need all that detail”
In Theorem proving
• Narrows (Amarel,
1986)
• Master variables
(Crawford 1995)
• Back doors
(Selman 2002).
In Software Eng.
• Saturation in
mutation testing
(Budd, 1980 and
many others
In Computer
Graphics
In Machine learning
• Variable subset
selection
(Kohavi, 1997)
• Instance selection
(Chen, 1975)
• Active learning
13
src= tiny.cc/gale15code
slides= tiny.cc/gale15
This talk
• What is search-based SE?
• Why use less CPU for SBSE?
• How to use less CPU
– Refactor the optimizer:
– Add in some data mining
• Experimental results
• Related Work
• Future work
• A challenge question:
– Are we making this too hard?
ai4se.net
14
src= tiny.cc/gale15code
slides= tiny.cc/gale15
Objectives = evaluate(decisions)
Generations *( Selection + Evaluation * Generation)
G * ( O(N2) + E * O(1)*N )
How to use less CPU (for SBSE)
ai4se.net
15
src= tiny.cc/gale15code
slides= tiny.cc/gale15
Objectives = evaluate(decisions)
Generations * (Selection + Evaluation * Generation)
G * ( O(N2) + E * O(1)*N )
How to use less CPU (for SBSE)
Approximate the space
• k=2 divisive clustering
ai4se.net
16
src= tiny.cc/gale15code
slides= tiny.cc/gale15
Objectives = evaluate(decisions)
Generations * (Selection + Evaluation * Generation)
G * ( O(N2) + E * O(1)*N )
How to use less CPU (for SBSE)
Approximate the space
• k=2 divisive clustering
(X,Y)= 2 very distant points in O(2N)
ai4se.net
17
src= tiny.cc/gale15code
slides= tiny.cc/gale15
Objectives = evaluate(decisions)
Generations * (Selection + Evaluation * Generation)
G * ( O(N2) + E * O(1)*N )
How to use less CPU (for SBSE)
Approximate the space
• k=2 divisive clustering
(X,Y)= 2 very distant points in O(2N)
Evaluate only (X,Y)
ai4se.net
18
src= tiny.cc/gale15code
slides= tiny.cc/gale15
Objectives = evaluate(decisions)
Generations * (Selection + Evaluation * Generation)
G * ( O(N2) + E * O(1)*N )
How to use less CPU (for SBSE)
Approximate the space
• k=2 divisive clustering
(X,Y)= 2 very distant points in O(2N)
Evaluate only (X,Y)
If better(X,Y)
• If size(cluster) > sqrt(N)
– Split, recurse on better half
– E.g. cull red
• Else, push points towards X
– E.g. push orange
ai4se.net
19
src= tiny.cc/gale15code
slides= tiny.cc/gale15
Objectives = evaluate(decisions)
Generations * (Selection + Evaluation * Generation)
G * ( O(N2) + E * O(1)*N )
How to use less CPU (for SBSE)
Red is
culled
Approximate the space
• k=2 divisive clustering
(X,Y)= 2 very distant points in O(2N)
Evaluate only (X,Y)
If better(X,Y)
• If size(cluster) > sqrt(N)
– Split, recurse on better half
– E.g. cull red
• Else, push points towards X
– E.g. push orange
ai4se.net
20
src= tiny.cc/gale15code
slides= tiny.cc/gale15
e.g. orange points
get pushed
this way
Objectives = evaluate(decisions)
Generations * (Selection + Evaluation * Generation)
G * ( O(N2) + E * O(1)*N )
How to use less CPU (for SBSE)
Red is
culled
Approximate the space
• k=2 divisive clustering
(X,Y)= 2 very distant points in O(2N)
Evaluate only (X,Y)
If better(X,Y)
• If size(cluster) > sqrt(N)
– Split, recurse on better half
– E.g. cull red
• Else, push points towards X
– E.g. push orange
ai4se.net
21
src= tiny.cc/gale15code
slides= tiny.cc/gale15
e.g. orange points
get pushed
this way
Objectives = evaluate(decisions)
Generations * (Selection + Evaluation * Generation)
G * O(N2) + E * O(1)*N
How to use less CPU (for SBSE)
Red is
culled
g * ( O(N) + log( E * O(1) * N))
Approximate the space
• k=2 divisive clustering
(X,Y)= 2 very distant points in O(2N)
Evaluate only (X,Y)
If better(X,Y)
• If size(cluster) > sqrt(N)
– Split, recurse on better half
– E.g. cull red
• Else, push points towards X
– E.g. push orange
ai4se.net
src= tiny.cc/gale15code
slides= tiny.cc/gale15
23
src= tiny.cc/gale15code
slides= tiny.cc/gale15
GALE’s clustering = fast analog for PCA
(so GALE is a heuristic spectral learner)
23ai4se.net
24
src= tiny.cc/gale15code
slides= tiny.cc/gale15
This talk
• What is search-based SE?
• Why use less CPU for SBSE?
• How to use less CPU
– Refactor the optimizer:
– Add in some data mining
• Experimental results
• Related Work
• Future work
• A challenge question:
– Are we making this too hard?
24ai4se.net
25ai4se.net
src= tiny.cc/gale15code
slides= tiny.cc/gale15
Sample models
Benchmark suites (small)
• The usual suspects: goo.gl/FTyhkJ
– 2-3 line equations
– Fonseca, Schaffer, woBar.
Golinski,
• Also, from goo.gl/w98wxu
– The ZDT suite :
– The DTLZ suite
SE models
• On-line at: goo.gl/nv2AVK
– XOMO: goo.gl/tY4nLu COCOMO
software effort estimator + defect
prediction + risk advisor
– POM3: goo.gl/RMxWC Agile teams
prioritizing tasks
• Tasks costs and utility may
subsequently change
• Teams depend on products from
other teams
• Internal NASA models:
– CDA: goo.gl/wLVrYA
• NASA’s requirements models for
human avionics
26ai4se.net
src= tiny.cc/gale15code
slides= tiny.cc/gale15
Comparison algorithms
What we used (in paper)
• NSGA-II (of course)
• SPEA2
• Selected from Sayyad et al’s
ICSE’13 survey of “usually
used MOEAs in SE”
Not IBEA:
– BTW, I don’t like IBEA, just its
continuous domination
function
– Used in GALE
Since paper
• Differential evolution
• MOEA/D
• ?NSGA III
– Some quirky “bunching
problems”
27
src= tiny.cc/gale15code
slides= tiny.cc/gale15
GALE: one of the best, far fewer evals
Gray: stats tests: as good as the best
ai4se.net
28
src= tiny.cc/gale15code
slides= tiny.cc/gale15
For small models, not much slower
For big models, 100 times faster
ai4se.net
29
src= tiny.cc/gale15code
slides= tiny.cc/gale15
On big models, GALE does very well
NASA’s requirements
models for human
avionics
• GALE: 4 mins
• NSGA-II: 8 hours
ai4se.net
30
src= tiny.cc/gale15code
slides= tiny.cc/gale15
DTLZ1:from 2 to 8 goals
ai4se.net
31
src= tiny.cc/gale15code
slides= tiny.cc/gale15
This talk
• What is search-based SE?
• Why use less CPU for SBSE?
• How to use less CPU
– Refactor the optimizer:
– Add in some data mining
• Experimental results
• Related Work
• Future work
• A challenge question:
– Are we making this too hard?
ai4se.net
32ai4se.net
src= tiny.cc/gale15code
slides= tiny.cc/gale15
Related work (more)
• Active learning [8]
– Don’t evaluate all,
– Just the most interesting
• Kamvar et al. 2003 [33]
– Spectral learning
• Boley , PDDP 1998 [34]
– Classification, recursive
descent on PCA component
– O(N2), not O(N)
• SPEA2, NSGA-II, PSO, DE,
MOEA/D, Tabu..
– All O(N) evaluations
• Various local search
methods (Peng [40])
– None known in SE
– None boasting GALE’s
reduced runtimes
• Response surface methods
Zuluaga [8]
– Parametric assumptions
about Pareto frontier
– Active learning
[X] = reference
in paper
33
src= tiny.cc/gale15code
slides= tiny.cc/gale15
This talk
• What is search-based SE?
• Why use less CPU SBSE?
• How to use less CPU
– Refactor the optimizer:
– Add in some data mining
• Experimental results
• Related Work
• Future work
• A challenge question:
– Are we making this too hard?
ai4se.net
34ai4se.net
src= tiny.cc/gale15code
slides= tiny.cc/gale15
Future work
More Models
• Siegmund & Apel’s runtime
configuration models
• Rungta’s NASA models of
space pilots flying MARS
missions
• 100s of Horkoff’s softgoal
models
• Software product lines
More Tool Building
• Explanation systems
– Complex MOEA tasks solved
by reflecting on only a few
dozen examples
– Human in the loop guidance
for the inference?
• There remains one loophole
GALE did not exploit
– So after GALE comes STORM,
– Work in progress
35
src= tiny.cc/gale15code
slides= tiny.cc/gale15
This talk
• What is search-based SE?
• Why use less CPU for SBSE?
• How to use less CPU
– Refactor the optimizer:
– Add in some data mining
• Experimental results
• Related Work
• Future work
• A challenge question:
– Are we making this too hard?
ai4se.net
37
src= tiny.cc/gale15code
slides= tiny.cc/gale15
GALE’s dangerous idea
• Simple approximations exist for seemingly complex problems.
• Researchers jump to the
complex before exploring
the simpler.
• Test supposedly sophisticated
vs simpler alternates (the
straw man).
• My career: “my straw
don’t burn”
ai4se.net
Slides: tiny.cc/gale15
Software: tiny.cc/gale15code
ai4se.net

More Related Content

What's hot

Andres Rodriguez at AI Frontiers: Catalyzing Deep Learning's Impact in the En...
Andres Rodriguez at AI Frontiers: Catalyzing Deep Learning's Impact in the En...Andres Rodriguez at AI Frontiers: Catalyzing Deep Learning's Impact in the En...
Andres Rodriguez at AI Frontiers: Catalyzing Deep Learning's Impact in the En...AI Frontiers
 
[Webinar] How Big Data and Machine Learning Are Transforming ITSM
[Webinar] How Big Data and Machine Learning Are Transforming ITSM[Webinar] How Big Data and Machine Learning Are Transforming ITSM
[Webinar] How Big Data and Machine Learning Are Transforming ITSMSunView Software, Inc.
 
Towards Mining Software Repositories Research that Matters
Towards Mining Software Repositories Research that MattersTowards Mining Software Repositories Research that Matters
Towards Mining Software Repositories Research that MattersTao Xie
 
Intro to Machine Learning
Intro to Machine LearningIntro to Machine Learning
Intro to Machine LearningCorey Chivers
 
Agile data science
Agile data scienceAgile data science
Agile data scienceJoel Horwitz
 
Software Mining and Software Datasets
Software Mining and Software DatasetsSoftware Mining and Software Datasets
Software Mining and Software DatasetsTao Xie
 
Deep Learning Use Cases - Data Science Pop-up Seattle
Deep Learning Use Cases - Data Science Pop-up SeattleDeep Learning Use Cases - Data Science Pop-up Seattle
Deep Learning Use Cases - Data Science Pop-up SeattleDomino Data Lab
 
machine learning in the age of big data: new approaches and business applicat...
machine learning in the age of big data: new approaches and business applicat...machine learning in the age of big data: new approaches and business applicat...
machine learning in the age of big data: new approaches and business applicat...Armando Vieira
 
Secure Because Math: A Deep-Dive on Machine Learning-Based Monitoring (#Secur...
Secure Because Math: A Deep-Dive on Machine Learning-Based Monitoring (#Secur...Secure Because Math: A Deep-Dive on Machine Learning-Based Monitoring (#Secur...
Secure Because Math: A Deep-Dive on Machine Learning-Based Monitoring (#Secur...Alex Pinto
 
ISEC'18 Tutorial: Research Methodology on Pursuing Impact-Driven Research
ISEC'18 Tutorial: Research Methodology on Pursuing Impact-Driven ResearchISEC'18 Tutorial: Research Methodology on Pursuing Impact-Driven Research
ISEC'18 Tutorial: Research Methodology on Pursuing Impact-Driven ResearchTao Xie
 
ACM Chicago March 2019 meeting: Software Engineering and AI - Prof. Tao Xie, ...
ACM Chicago March 2019 meeting: Software Engineering and AI - Prof. Tao Xie, ...ACM Chicago March 2019 meeting: Software Engineering and AI - Prof. Tao Xie, ...
ACM Chicago March 2019 meeting: Software Engineering and AI - Prof. Tao Xie, ...ACM Chicago
 
Data Science Reinvents Learning?
Data Science Reinvents Learning?Data Science Reinvents Learning?
Data Science Reinvents Learning?Paco Nathan
 
Data Science, Machine Learning, and H2O
Data Science, Machine Learning, and H2OData Science, Machine Learning, and H2O
Data Science, Machine Learning, and H2OSri Ambati
 
Full-stack Data Scientist
Full-stack Data ScientistFull-stack Data Scientist
Full-stack Data ScientistAlexey Grigorev
 
Josh Wills, MLconf 2013
Josh Wills, MLconf 2013Josh Wills, MLconf 2013
Josh Wills, MLconf 2013MLconf
 
Data Science 101
Data Science 101Data Science 101
Data Science 101odsc
 
Data Science in Future Tense
Data Science in Future TenseData Science in Future Tense
Data Science in Future TensePaco Nathan
 
Sharing is Caring: Understanding and Measuring Threat Intelligence Sharing Ef...
Sharing is Caring: Understanding and Measuring Threat Intelligence Sharing Ef...Sharing is Caring: Understanding and Measuring Threat Intelligence Sharing Ef...
Sharing is Caring: Understanding and Measuring Threat Intelligence Sharing Ef...Alex Pinto
 
BSidesLV 2013 - Using Machine Learning to Support Information Security
BSidesLV 2013 - Using Machine Learning to Support Information SecurityBSidesLV 2013 - Using Machine Learning to Support Information Security
BSidesLV 2013 - Using Machine Learning to Support Information SecurityAlex Pinto
 
Determining the Fit and Impact of CTI Indicators on Your Monitoring Pipeline ...
Determining the Fit and Impact of CTI Indicators on Your Monitoring Pipeline ...Determining the Fit and Impact of CTI Indicators on Your Monitoring Pipeline ...
Determining the Fit and Impact of CTI Indicators on Your Monitoring Pipeline ...Alex Pinto
 

What's hot (20)

Andres Rodriguez at AI Frontiers: Catalyzing Deep Learning's Impact in the En...
Andres Rodriguez at AI Frontiers: Catalyzing Deep Learning's Impact in the En...Andres Rodriguez at AI Frontiers: Catalyzing Deep Learning's Impact in the En...
Andres Rodriguez at AI Frontiers: Catalyzing Deep Learning's Impact in the En...
 
[Webinar] How Big Data and Machine Learning Are Transforming ITSM
[Webinar] How Big Data and Machine Learning Are Transforming ITSM[Webinar] How Big Data and Machine Learning Are Transforming ITSM
[Webinar] How Big Data and Machine Learning Are Transforming ITSM
 
Towards Mining Software Repositories Research that Matters
Towards Mining Software Repositories Research that MattersTowards Mining Software Repositories Research that Matters
Towards Mining Software Repositories Research that Matters
 
Intro to Machine Learning
Intro to Machine LearningIntro to Machine Learning
Intro to Machine Learning
 
Agile data science
Agile data scienceAgile data science
Agile data science
 
Software Mining and Software Datasets
Software Mining and Software DatasetsSoftware Mining and Software Datasets
Software Mining and Software Datasets
 
Deep Learning Use Cases - Data Science Pop-up Seattle
Deep Learning Use Cases - Data Science Pop-up SeattleDeep Learning Use Cases - Data Science Pop-up Seattle
Deep Learning Use Cases - Data Science Pop-up Seattle
 
machine learning in the age of big data: new approaches and business applicat...
machine learning in the age of big data: new approaches and business applicat...machine learning in the age of big data: new approaches and business applicat...
machine learning in the age of big data: new approaches and business applicat...
 
Secure Because Math: A Deep-Dive on Machine Learning-Based Monitoring (#Secur...
Secure Because Math: A Deep-Dive on Machine Learning-Based Monitoring (#Secur...Secure Because Math: A Deep-Dive on Machine Learning-Based Monitoring (#Secur...
Secure Because Math: A Deep-Dive on Machine Learning-Based Monitoring (#Secur...
 
ISEC'18 Tutorial: Research Methodology on Pursuing Impact-Driven Research
ISEC'18 Tutorial: Research Methodology on Pursuing Impact-Driven ResearchISEC'18 Tutorial: Research Methodology on Pursuing Impact-Driven Research
ISEC'18 Tutorial: Research Methodology on Pursuing Impact-Driven Research
 
ACM Chicago March 2019 meeting: Software Engineering and AI - Prof. Tao Xie, ...
ACM Chicago March 2019 meeting: Software Engineering and AI - Prof. Tao Xie, ...ACM Chicago March 2019 meeting: Software Engineering and AI - Prof. Tao Xie, ...
ACM Chicago March 2019 meeting: Software Engineering and AI - Prof. Tao Xie, ...
 
Data Science Reinvents Learning?
Data Science Reinvents Learning?Data Science Reinvents Learning?
Data Science Reinvents Learning?
 
Data Science, Machine Learning, and H2O
Data Science, Machine Learning, and H2OData Science, Machine Learning, and H2O
Data Science, Machine Learning, and H2O
 
Full-stack Data Scientist
Full-stack Data ScientistFull-stack Data Scientist
Full-stack Data Scientist
 
Josh Wills, MLconf 2013
Josh Wills, MLconf 2013Josh Wills, MLconf 2013
Josh Wills, MLconf 2013
 
Data Science 101
Data Science 101Data Science 101
Data Science 101
 
Data Science in Future Tense
Data Science in Future TenseData Science in Future Tense
Data Science in Future Tense
 
Sharing is Caring: Understanding and Measuring Threat Intelligence Sharing Ef...
Sharing is Caring: Understanding and Measuring Threat Intelligence Sharing Ef...Sharing is Caring: Understanding and Measuring Threat Intelligence Sharing Ef...
Sharing is Caring: Understanding and Measuring Threat Intelligence Sharing Ef...
 
BSidesLV 2013 - Using Machine Learning to Support Information Security
BSidesLV 2013 - Using Machine Learning to Support Information SecurityBSidesLV 2013 - Using Machine Learning to Support Information Security
BSidesLV 2013 - Using Machine Learning to Support Information Security
 
Determining the Fit and Impact of CTI Indicators on Your Monitoring Pipeline ...
Determining the Fit and Impact of CTI Indicators on Your Monitoring Pipeline ...Determining the Fit and Impact of CTI Indicators on Your Monitoring Pipeline ...
Determining the Fit and Impact of CTI Indicators on Your Monitoring Pipeline ...
 

Viewers also liked

In the age of Big Data, what role for Software Engineers?
In the age of Big Data, what role for Software Engineers?In the age of Big Data, what role for Software Engineers?
In the age of Big Data, what role for Software Engineers?CS, NcState
 
Mushroom tutorial http://rjdatamining.weebly.com
Mushroom tutorial http://rjdatamining.weebly.comMushroom tutorial http://rjdatamining.weebly.com
Mushroom tutorial http://rjdatamining.weebly.comrayborg
 
Project 3 mushrooms
Project 3 mushroomsProject 3 mushrooms
Project 3 mushroomsCS, NcState
 
Asheville school mushroom program
Asheville school mushroom programAsheville school mushroom program
Asheville school mushroom programlelawrence03
 
Mushroom culture
Mushroom cultureMushroom culture
Mushroom cultureelachakiel
 
Mushroom Cultivation and Marketing
Mushroom Cultivation and MarketingMushroom Cultivation and Marketing
Mushroom Cultivation and MarketingElisaMendelsohn
 
Final ppt on mushroom
Final ppt on mushroomFinal ppt on mushroom
Final ppt on mushroomAjay Patidar
 
Mushroom Cultivation
Mushroom CultivationMushroom Cultivation
Mushroom CultivationGowri Prabhu
 

Viewers also liked (12)

In the age of Big Data, what role for Software Engineers?
In the age of Big Data, what role for Software Engineers?In the age of Big Data, what role for Software Engineers?
In the age of Big Data, what role for Software Engineers?
 
Mushroom tutorial http://rjdatamining.weebly.com
Mushroom tutorial http://rjdatamining.weebly.comMushroom tutorial http://rjdatamining.weebly.com
Mushroom tutorial http://rjdatamining.weebly.com
 
Project 3 mushrooms
Project 3 mushroomsProject 3 mushrooms
Project 3 mushrooms
 
Asheville school mushroom program
Asheville school mushroom programAsheville school mushroom program
Asheville school mushroom program
 
Different types of mushrooms
Different types of mushroomsDifferent types of mushrooms
Different types of mushrooms
 
Mushroom business plan
Mushroom business planMushroom business plan
Mushroom business plan
 
Mushroom culture
Mushroom cultureMushroom culture
Mushroom culture
 
Mushroom Cultivation and Marketing
Mushroom Cultivation and MarketingMushroom Cultivation and Marketing
Mushroom Cultivation and Marketing
 
Mushroom cultivation
Mushroom cultivationMushroom cultivation
Mushroom cultivation
 
Final ppt on mushroom
Final ppt on mushroomFinal ppt on mushroom
Final ppt on mushroom
 
Mushroom Cultivation
Mushroom CultivationMushroom Cultivation
Mushroom Cultivation
 
mushroom culture
mushroom culturemushroom culture
mushroom culture
 

Similar to GALE: Geometric active learning for Search-Based Software Engineering

Contributions to the Efficient Use of General Purpose Coprocessors: KDE as Ca...
Contributions to the Efficient Use of General Purpose Coprocessors: KDE as Ca...Contributions to the Efficient Use of General Purpose Coprocessors: KDE as Ca...
Contributions to the Efficient Use of General Purpose Coprocessors: KDE as Ca...Unai Lopez-Novoa
 
Salt Identification Challenge
Salt Identification ChallengeSalt Identification Challenge
Salt Identification Challengekenluck2001
 
SQL Query Optimization: Why Is It So Hard to Get Right?
SQL Query Optimization: Why Is It So Hard to Get Right?SQL Query Optimization: Why Is It So Hard to Get Right?
SQL Query Optimization: Why Is It So Hard to Get Right?Brent Ozar
 
It Does What You Say, Not What You Mean: Lessons From A Decade of Program Repair
It Does What You Say, Not What You Mean: Lessons From A Decade of Program RepairIt Does What You Say, Not What You Mean: Lessons From A Decade of Program Repair
It Does What You Say, Not What You Mean: Lessons From A Decade of Program RepairClaire Le Goues
 
Database and application performance vivek sharma
Database and application performance vivek sharmaDatabase and application performance vivek sharma
Database and application performance vivek sharmaaioughydchapter
 
The Other HPC: High Productivity Computing
The Other HPC: High Productivity ComputingThe Other HPC: High Productivity Computing
The Other HPC: High Productivity ComputingUniversity of Washington
 
On the value of Sampling and Pruning for SBSE
On the value of Sampling and Pruning for SBSEOn the value of Sampling and Pruning for SBSE
On the value of Sampling and Pruning for SBSEJianfeng Chen
 
Hadoop in Data Warehousing
Hadoop in Data WarehousingHadoop in Data Warehousing
Hadoop in Data WarehousingAlexey Grigorev
 
Making Sense of Spark Performance-(Kay Ousterhout, UC Berkeley)
Making Sense of Spark Performance-(Kay Ousterhout, UC Berkeley)Making Sense of Spark Performance-(Kay Ousterhout, UC Berkeley)
Making Sense of Spark Performance-(Kay Ousterhout, UC Berkeley)Spark Summit
 
Optimization of Continuous Queries in Federated Database and Stream Processin...
Optimization of Continuous Queries in Federated Database and Stream Processin...Optimization of Continuous Queries in Federated Database and Stream Processin...
Optimization of Continuous Queries in Federated Database and Stream Processin...Zbigniew Jerzak
 
PyData 2015 Keynote: "A Systems View of Machine Learning"
PyData 2015 Keynote: "A Systems View of Machine Learning" PyData 2015 Keynote: "A Systems View of Machine Learning"
PyData 2015 Keynote: "A Systems View of Machine Learning" Joshua Bloom
 
CocomoModels MGK .ppt
CocomoModels MGK .pptCocomoModels MGK .ppt
CocomoModels MGK .pptssuser3d1dad3
 
Using Deep Learning on Apache Spark to Diagnose Thoracic Pathology from Chest...
Using Deep Learning on Apache Spark to Diagnose Thoracic Pathology from Chest...Using Deep Learning on Apache Spark to Diagnose Thoracic Pathology from Chest...
Using Deep Learning on Apache Spark to Diagnose Thoracic Pathology from Chest...Databricks
 
Large-scale Recommendation Systems on Just a PC
Large-scale Recommendation Systems on Just a PCLarge-scale Recommendation Systems on Just a PC
Large-scale Recommendation Systems on Just a PCAapo Kyrölä
 
Macy's: Changing Engines in Mid-Flight
Macy's: Changing Engines in Mid-FlightMacy's: Changing Engines in Mid-Flight
Macy's: Changing Engines in Mid-FlightDataStax Academy
 
SoC HPC: Design, Optimization, and Application to Algorithmic Trading
SoC HPC: Design, Optimization, and Application to Algorithmic TradingSoC HPC: Design, Optimization, and Application to Algorithmic Trading
SoC HPC: Design, Optimization, and Application to Algorithmic TradingMark Delgado
 
Certified Reasoning for Automated Verification
Certified Reasoning for Automated VerificationCertified Reasoning for Automated Verification
Certified Reasoning for Automated VerificationAsankhaya Sharma
 
An early look at the LDBC Social Network Benchmark's Business Intelligence wo...
An early look at the LDBC Social Network Benchmark's Business Intelligence wo...An early look at the LDBC Social Network Benchmark's Business Intelligence wo...
An early look at the LDBC Social Network Benchmark's Business Intelligence wo...Gábor Szárnyas
 
ntcir14centre-overview
ntcir14centre-overviewntcir14centre-overview
ntcir14centre-overviewTetsuya Sakai
 

Similar to GALE: Geometric active learning for Search-Based Software Engineering (20)

Contributions to the Efficient Use of General Purpose Coprocessors: KDE as Ca...
Contributions to the Efficient Use of General Purpose Coprocessors: KDE as Ca...Contributions to the Efficient Use of General Purpose Coprocessors: KDE as Ca...
Contributions to the Efficient Use of General Purpose Coprocessors: KDE as Ca...
 
Salt Identification Challenge
Salt Identification ChallengeSalt Identification Challenge
Salt Identification Challenge
 
SQL Query Optimization: Why Is It So Hard to Get Right?
SQL Query Optimization: Why Is It So Hard to Get Right?SQL Query Optimization: Why Is It So Hard to Get Right?
SQL Query Optimization: Why Is It So Hard to Get Right?
 
It Does What You Say, Not What You Mean: Lessons From A Decade of Program Repair
It Does What You Say, Not What You Mean: Lessons From A Decade of Program RepairIt Does What You Say, Not What You Mean: Lessons From A Decade of Program Repair
It Does What You Say, Not What You Mean: Lessons From A Decade of Program Repair
 
Database and application performance vivek sharma
Database and application performance vivek sharmaDatabase and application performance vivek sharma
Database and application performance vivek sharma
 
The Other HPC: High Productivity Computing
The Other HPC: High Productivity ComputingThe Other HPC: High Productivity Computing
The Other HPC: High Productivity Computing
 
On the value of Sampling and Pruning for SBSE
On the value of Sampling and Pruning for SBSEOn the value of Sampling and Pruning for SBSE
On the value of Sampling and Pruning for SBSE
 
Hadoop in Data Warehousing
Hadoop in Data WarehousingHadoop in Data Warehousing
Hadoop in Data Warehousing
 
Making Sense of Spark Performance-(Kay Ousterhout, UC Berkeley)
Making Sense of Spark Performance-(Kay Ousterhout, UC Berkeley)Making Sense of Spark Performance-(Kay Ousterhout, UC Berkeley)
Making Sense of Spark Performance-(Kay Ousterhout, UC Berkeley)
 
Optimization of Continuous Queries in Federated Database and Stream Processin...
Optimization of Continuous Queries in Federated Database and Stream Processin...Optimization of Continuous Queries in Federated Database and Stream Processin...
Optimization of Continuous Queries in Federated Database and Stream Processin...
 
Available HPC resources at CSUC
Available HPC resources at CSUCAvailable HPC resources at CSUC
Available HPC resources at CSUC
 
PyData 2015 Keynote: "A Systems View of Machine Learning"
PyData 2015 Keynote: "A Systems View of Machine Learning" PyData 2015 Keynote: "A Systems View of Machine Learning"
PyData 2015 Keynote: "A Systems View of Machine Learning"
 
CocomoModels MGK .ppt
CocomoModels MGK .pptCocomoModels MGK .ppt
CocomoModels MGK .ppt
 
Using Deep Learning on Apache Spark to Diagnose Thoracic Pathology from Chest...
Using Deep Learning on Apache Spark to Diagnose Thoracic Pathology from Chest...Using Deep Learning on Apache Spark to Diagnose Thoracic Pathology from Chest...
Using Deep Learning on Apache Spark to Diagnose Thoracic Pathology from Chest...
 
Large-scale Recommendation Systems on Just a PC
Large-scale Recommendation Systems on Just a PCLarge-scale Recommendation Systems on Just a PC
Large-scale Recommendation Systems on Just a PC
 
Macy's: Changing Engines in Mid-Flight
Macy's: Changing Engines in Mid-FlightMacy's: Changing Engines in Mid-Flight
Macy's: Changing Engines in Mid-Flight
 
SoC HPC: Design, Optimization, and Application to Algorithmic Trading
SoC HPC: Design, Optimization, and Application to Algorithmic TradingSoC HPC: Design, Optimization, and Application to Algorithmic Trading
SoC HPC: Design, Optimization, and Application to Algorithmic Trading
 
Certified Reasoning for Automated Verification
Certified Reasoning for Automated VerificationCertified Reasoning for Automated Verification
Certified Reasoning for Automated Verification
 
An early look at the LDBC Social Network Benchmark's Business Intelligence wo...
An early look at the LDBC Social Network Benchmark's Business Intelligence wo...An early look at the LDBC Social Network Benchmark's Business Intelligence wo...
An early look at the LDBC Social Network Benchmark's Business Intelligence wo...
 
ntcir14centre-overview
ntcir14centre-overviewntcir14centre-overview
ntcir14centre-overview
 

More from CS, NcState

Lexisnexis june9
Lexisnexis june9Lexisnexis june9
Lexisnexis june9CS, NcState
 
Welcome to ICSE NIER’15 (new ideas and emerging results).
Welcome to ICSE NIER’15 (new ideas and emerging results).Welcome to ICSE NIER’15 (new ideas and emerging results).
Welcome to ICSE NIER’15 (new ideas and emerging results).CS, NcState
 
Icse15 Tech-briefing Data Science
Icse15 Tech-briefing Data ScienceIcse15 Tech-briefing Data Science
Icse15 Tech-briefing Data ScienceCS, NcState
 
Kits to Find the Bits that Fits
Kits to Find  the Bits that Fits Kits to Find  the Bits that Fits
Kits to Find the Bits that Fits CS, NcState
 
Ai4se lab template
Ai4se lab templateAi4se lab template
Ai4se lab templateCS, NcState
 
Automated Software Enging, Fall 2015, NCSU
Automated Software Enging, Fall 2015, NCSUAutomated Software Enging, Fall 2015, NCSU
Automated Software Enging, Fall 2015, NCSUCS, NcState
 
Requirements Engineering
Requirements EngineeringRequirements Engineering
Requirements EngineeringCS, NcState
 
172529main ken and_tim_software_assurance_research_at_west_virginia
172529main ken and_tim_software_assurance_research_at_west_virginia172529main ken and_tim_software_assurance_research_at_west_virginia
172529main ken and_tim_software_assurance_research_at_west_virginiaCS, NcState
 
Automated Software Engineering
Automated Software EngineeringAutomated Software Engineering
Automated Software EngineeringCS, NcState
 
Next Generation “Treatment Learning” (finding the diamonds in the dust)
Next Generation “Treatment Learning” (finding the diamonds in the dust)Next Generation “Treatment Learning” (finding the diamonds in the dust)
Next Generation “Treatment Learning” (finding the diamonds in the dust)CS, NcState
 
Tim Menzies, directions in Data Science
Tim Menzies, directions in Data ScienceTim Menzies, directions in Data Science
Tim Menzies, directions in Data ScienceCS, NcState
 
Dagstuhl14 intro-v1
Dagstuhl14 intro-v1Dagstuhl14 intro-v1
Dagstuhl14 intro-v1CS, NcState
 
The Art and Science of Analyzing Software Data
The Art and Science of Analyzing Software DataThe Art and Science of Analyzing Software Data
The Art and Science of Analyzing Software DataCS, NcState
 
What Metrics Matter?
What Metrics Matter? What Metrics Matter?
What Metrics Matter? CS, NcState
 
Sayyad slides ase13_v4
Sayyad slides ase13_v4Sayyad slides ase13_v4
Sayyad slides ase13_v4CS, NcState
 
Warning: don't do CS
Warning: don't do CSWarning: don't do CS
Warning: don't do CSCS, NcState
 
How to do better experiments in SE
How to do better experiments in SEHow to do better experiments in SE
How to do better experiments in SECS, NcState
 

More from CS, NcState (20)

Lexisnexis june9
Lexisnexis june9Lexisnexis june9
Lexisnexis june9
 
Welcome to ICSE NIER’15 (new ideas and emerging results).
Welcome to ICSE NIER’15 (new ideas and emerging results).Welcome to ICSE NIER’15 (new ideas and emerging results).
Welcome to ICSE NIER’15 (new ideas and emerging results).
 
Icse15 Tech-briefing Data Science
Icse15 Tech-briefing Data ScienceIcse15 Tech-briefing Data Science
Icse15 Tech-briefing Data Science
 
Kits to Find the Bits that Fits
Kits to Find  the Bits that Fits Kits to Find  the Bits that Fits
Kits to Find the Bits that Fits
 
Ai4se lab template
Ai4se lab templateAi4se lab template
Ai4se lab template
 
Automated Software Enging, Fall 2015, NCSU
Automated Software Enging, Fall 2015, NCSUAutomated Software Enging, Fall 2015, NCSU
Automated Software Enging, Fall 2015, NCSU
 
Requirements Engineering
Requirements EngineeringRequirements Engineering
Requirements Engineering
 
172529main ken and_tim_software_assurance_research_at_west_virginia
172529main ken and_tim_software_assurance_research_at_west_virginia172529main ken and_tim_software_assurance_research_at_west_virginia
172529main ken and_tim_software_assurance_research_at_west_virginia
 
Automated Software Engineering
Automated Software EngineeringAutomated Software Engineering
Automated Software Engineering
 
Next Generation “Treatment Learning” (finding the diamonds in the dust)
Next Generation “Treatment Learning” (finding the diamonds in the dust)Next Generation “Treatment Learning” (finding the diamonds in the dust)
Next Generation “Treatment Learning” (finding the diamonds in the dust)
 
Tim Menzies, directions in Data Science
Tim Menzies, directions in Data ScienceTim Menzies, directions in Data Science
Tim Menzies, directions in Data Science
 
Goldrush
GoldrushGoldrush
Goldrush
 
Dagstuhl14 intro-v1
Dagstuhl14 intro-v1Dagstuhl14 intro-v1
Dagstuhl14 intro-v1
 
Know thy tools
Know thy toolsKnow thy tools
Know thy tools
 
The Art and Science of Analyzing Software Data
The Art and Science of Analyzing Software DataThe Art and Science of Analyzing Software Data
The Art and Science of Analyzing Software Data
 
What Metrics Matter?
What Metrics Matter? What Metrics Matter?
What Metrics Matter?
 
Sayyad slides ase13_v4
Sayyad slides ase13_v4Sayyad slides ase13_v4
Sayyad slides ase13_v4
 
Ase2013
Ase2013Ase2013
Ase2013
 
Warning: don't do CS
Warning: don't do CSWarning: don't do CS
Warning: don't do CS
 
How to do better experiments in SE
How to do better experiments in SEHow to do better experiments in SE
How to do better experiments in SE
 

Recently uploaded

Network Enhancements on BitVisor for BitVisor Summit 12
Network Enhancements on BitVisor for BitVisor Summit 12Network Enhancements on BitVisor for BitVisor Summit 12
Network Enhancements on BitVisor for BitVisor Summit 12cjchen22
 
SOFTWARE ESTIMATION COCOMO AND FP CALCULATION
SOFTWARE ESTIMATION COCOMO AND FP CALCULATIONSOFTWARE ESTIMATION COCOMO AND FP CALCULATION
SOFTWARE ESTIMATION COCOMO AND FP CALCULATIONSneha Padhiar
 
Defining the Clouds for entriprises.pptx
Defining the Clouds for entriprises.pptxDefining the Clouds for entriprises.pptx
Defining the Clouds for entriprises.pptxAshwiniTodkar4
 
Road Development plans of India-Transportation Engineering
Road Development plans of India-Transportation EngineeringRoad Development plans of India-Transportation Engineering
Road Development plans of India-Transportation EngineeringSayli Madhale
 
Triangulation survey (Basic Mine Surveying)_MI10412MI.pptx
Triangulation survey (Basic Mine Surveying)_MI10412MI.pptxTriangulation survey (Basic Mine Surveying)_MI10412MI.pptx
Triangulation survey (Basic Mine Surveying)_MI10412MI.pptxRomil Mishra
 
input buffering in lexical analysis in CD
input buffering in lexical analysis in CDinput buffering in lexical analysis in CD
input buffering in lexical analysis in CDHeadOfDepartmentComp1
 
Feasibility analysis and modeling of a solar hybrid system for residential el...
Feasibility analysis and modeling of a solar hybrid system for residential el...Feasibility analysis and modeling of a solar hybrid system for residential el...
Feasibility analysis and modeling of a solar hybrid system for residential el...IJECEIAES
 
Javier_Fernandez_CARS_workshop_presentation.pptx
Javier_Fernandez_CARS_workshop_presentation.pptxJavier_Fernandez_CARS_workshop_presentation.pptx
Javier_Fernandez_CARS_workshop_presentation.pptxJavier Fernández Muñoz
 
AI Powered Ecover creator that creates everything that you want
AI Powered Ecover creator that creates everything that you wantAI Powered Ecover creator that creates everything that you want
AI Powered Ecover creator that creates everything that you wantsuja868966
 
Mohanakumar K R Davangere_3[1] 4d1 me.pptx
Mohanakumar K R Davangere_3[1] 4d1 me.pptxMohanakumar K R Davangere_3[1] 4d1 me.pptx
Mohanakumar K R Davangere_3[1] 4d1 me.pptxMohan377827
 
Introduction to Machine Learning Part1.pptx
Introduction to Machine Learning Part1.pptxIntroduction to Machine Learning Part1.pptx
Introduction to Machine Learning Part1.pptxPavan Mohan Neelamraju
 
Cost estimation approach: FP to COCOMO scenario based question
Cost estimation approach: FP to COCOMO scenario based questionCost estimation approach: FP to COCOMO scenario based question
Cost estimation approach: FP to COCOMO scenario based questionSneha Padhiar
 
LM7_ Embedded Sql and Dynamic SQL in dbms
LM7_ Embedded Sql and Dynamic SQL in dbmsLM7_ Embedded Sql and Dynamic SQL in dbms
LM7_ Embedded Sql and Dynamic SQL in dbmsBalaKrish12
 
Introduction of Object Oriented Programming Language using Java. .pptx
Introduction of Object Oriented Programming Language using Java. .pptxIntroduction of Object Oriented Programming Language using Java. .pptx
Introduction of Object Oriented Programming Language using Java. .pptxPoonam60376
 
Ece technical seminar topic for under graduate.pptx
Ece technical seminar topic for under graduate.pptxEce technical seminar topic for under graduate.pptx
Ece technical seminar topic for under graduate.pptxArjunPLinekaje
 
Madani.store - Planning - Interview Questions
Madani.store - Planning - Interview QuestionsMadani.store - Planning - Interview Questions
Madani.store - Planning - Interview QuestionsKarim Gaber
 
KCD Costa Rica 2024 - Nephio para parvulitos
KCD Costa Rica 2024 - Nephio para parvulitosKCD Costa Rica 2024 - Nephio para parvulitos
KCD Costa Rica 2024 - Nephio para parvulitosVictor Morales
 
LEA's chemistry of cement and concrete - 2019.pdf
LEA's chemistry of cement and concrete - 2019.pdfLEA's chemistry of cement and concrete - 2019.pdf
LEA's chemistry of cement and concrete - 2019.pdfJurgen Kola
 
Theory of Machine Notes / Lecture Material .pdf
Theory of Machine Notes / Lecture Material .pdfTheory of Machine Notes / Lecture Material .pdf
Theory of Machine Notes / Lecture Material .pdfShreyas Pandit
 

Recently uploaded (20)

Network Enhancements on BitVisor for BitVisor Summit 12
Network Enhancements on BitVisor for BitVisor Summit 12Network Enhancements on BitVisor for BitVisor Summit 12
Network Enhancements on BitVisor for BitVisor Summit 12
 
SOFTWARE ESTIMATION COCOMO AND FP CALCULATION
SOFTWARE ESTIMATION COCOMO AND FP CALCULATIONSOFTWARE ESTIMATION COCOMO AND FP CALCULATION
SOFTWARE ESTIMATION COCOMO AND FP CALCULATION
 
Defining the Clouds for entriprises.pptx
Defining the Clouds for entriprises.pptxDefining the Clouds for entriprises.pptx
Defining the Clouds for entriprises.pptx
 
Road Development plans of India-Transportation Engineering
Road Development plans of India-Transportation EngineeringRoad Development plans of India-Transportation Engineering
Road Development plans of India-Transportation Engineering
 
Triangulation survey (Basic Mine Surveying)_MI10412MI.pptx
Triangulation survey (Basic Mine Surveying)_MI10412MI.pptxTriangulation survey (Basic Mine Surveying)_MI10412MI.pptx
Triangulation survey (Basic Mine Surveying)_MI10412MI.pptx
 
input buffering in lexical analysis in CD
input buffering in lexical analysis in CDinput buffering in lexical analysis in CD
input buffering in lexical analysis in CD
 
Feasibility analysis and modeling of a solar hybrid system for residential el...
Feasibility analysis and modeling of a solar hybrid system for residential el...Feasibility analysis and modeling of a solar hybrid system for residential el...
Feasibility analysis and modeling of a solar hybrid system for residential el...
 
Javier_Fernandez_CARS_workshop_presentation.pptx
Javier_Fernandez_CARS_workshop_presentation.pptxJavier_Fernandez_CARS_workshop_presentation.pptx
Javier_Fernandez_CARS_workshop_presentation.pptx
 
ASME-B31.4-2019-estandar para diseño de ductos
ASME-B31.4-2019-estandar para diseño de ductosASME-B31.4-2019-estandar para diseño de ductos
ASME-B31.4-2019-estandar para diseño de ductos
 
AI Powered Ecover creator that creates everything that you want
AI Powered Ecover creator that creates everything that you wantAI Powered Ecover creator that creates everything that you want
AI Powered Ecover creator that creates everything that you want
 
Mohanakumar K R Davangere_3[1] 4d1 me.pptx
Mohanakumar K R Davangere_3[1] 4d1 me.pptxMohanakumar K R Davangere_3[1] 4d1 me.pptx
Mohanakumar K R Davangere_3[1] 4d1 me.pptx
 
Introduction to Machine Learning Part1.pptx
Introduction to Machine Learning Part1.pptxIntroduction to Machine Learning Part1.pptx
Introduction to Machine Learning Part1.pptx
 
Cost estimation approach: FP to COCOMO scenario based question
Cost estimation approach: FP to COCOMO scenario based questionCost estimation approach: FP to COCOMO scenario based question
Cost estimation approach: FP to COCOMO scenario based question
 
LM7_ Embedded Sql and Dynamic SQL in dbms
LM7_ Embedded Sql and Dynamic SQL in dbmsLM7_ Embedded Sql and Dynamic SQL in dbms
LM7_ Embedded Sql and Dynamic SQL in dbms
 
Introduction of Object Oriented Programming Language using Java. .pptx
Introduction of Object Oriented Programming Language using Java. .pptxIntroduction of Object Oriented Programming Language using Java. .pptx
Introduction of Object Oriented Programming Language using Java. .pptx
 
Ece technical seminar topic for under graduate.pptx
Ece technical seminar topic for under graduate.pptxEce technical seminar topic for under graduate.pptx
Ece technical seminar topic for under graduate.pptx
 
Madani.store - Planning - Interview Questions
Madani.store - Planning - Interview QuestionsMadani.store - Planning - Interview Questions
Madani.store - Planning - Interview Questions
 
KCD Costa Rica 2024 - Nephio para parvulitos
KCD Costa Rica 2024 - Nephio para parvulitosKCD Costa Rica 2024 - Nephio para parvulitos
KCD Costa Rica 2024 - Nephio para parvulitos
 
LEA's chemistry of cement and concrete - 2019.pdf
LEA's chemistry of cement and concrete - 2019.pdfLEA's chemistry of cement and concrete - 2019.pdf
LEA's chemistry of cement and concrete - 2019.pdf
 
Theory of Machine Notes / Lecture Material .pdf
Theory of Machine Notes / Lecture Material .pdfTheory of Machine Notes / Lecture Material .pdf
Theory of Machine Notes / Lecture Material .pdf
 

GALE: Geometric active learning for Search-Based Software Engineering

  • 1. 1 src= tiny.cc/gale15code slides= tiny.cc/gale15 GALE: Geometric active learning for Search-Based Software Engineering Joseph Krall, LoadIQ Tim Menzies, NC State Misty Davies, NASA Ames Sept 5, 2015 Slides: tiny.cc/gale15 Software: tiny.cc/gale15code 10th Joint Meeting of the European Software Engineering Conference and the ACM SIGSOFT Symposium on the Foundations of Software Engineering: FSE’15 ai4se.net
  • 2. 2 src= tiny.cc/gale15code slides= tiny.cc/gale15 This talk • What is search-based SE? • Why use less CPU for SBSE? • How to use less CPU – Refactor the optimizer: – Add in some data mining • Experimental results • Related Work • Future work • A challenge question: – Are we making this too hard? ai4se.net
  • 3.
  • 4. 4 src= tiny.cc/gale15code slides= tiny.cc/gale15 This talk • What is search-based SE? • Why use less CPU for SBSE? • How to use less CPU – Refactor the optimizer: – Add in some data mining • Experimental results • Related Work • Future work • A challenge question: – Are we making this too hard? ai4se.net
  • 5. 5ai4se.net src= tiny.cc/gale15code slides= tiny.cc/gale15 Q: What is Search-based SE? A: The future • Ye olde SE – Manually code up your understanding a domain – Struggle to understand that software • Search-based- model-based SE – Code up domain knowledge into a model – Explore that model – All models are wrong • But some are useful
  • 6. 6 src= tiny.cc/gale15code slides= tiny.cc/gale15 SBSE = everything 1. Requirements Menzies, Feather, Bagnall, Mansouri, Zhang 2. Transformation Cooper, Ryan, Schielke, Subramanian, Fatiregun, Williams 3.Effort prediction Aguilar-Ruiz, Burgess, Dolado, Lefley, Shepperd 4. Management Alba, Antoniol, Chicano, Di Pentam Greer, Ruhe 5. Heap allocation Cohen, Kooi, Srisa-an 6. Regression test Li, Yoo, Elbaum, Rothermel, Walcott, Soffa, Kampfhamer 7. SOA Canfora, Di Penta, Esposito, Villani 8. Refactoring Antoniol, Briand, Cinneide, O’Keeffe, Merlo, Seng, Tratt 9. Test Generation Alba, Binkley, Bottaci, Briand, Chicano, Clark, Cohen, Gutjahr, Harrold, Holcombe, Jones, Korel, Pargass, Reformat, Roper, McMinn, Michael, Sthamer, Tracy, Tonella,Xanthakis, Xiao, Wegener, Wilkins 10. Maintenance Antoniol, Lutz, Di Penta, Madhavi, Mancoridis, Mitchell, Swift 11. Model checking Alba, Chicano, Godefroid 12. Probing Cohen, Elbaum 13. UIOs Derderian, Guo, Hierons 14. Comprehension Gold, Li, Mahdavi 15. Protocols Alba, Clark, Jacob, Troya 16. Component sel Baker, Skaliotis, Steinhofel, Yoo 17. Agent Oriented Haas, Peysakhov, Sinclair, Shami, Mancoridisai4se.net
  • 7. 7 src= tiny.cc/gale15code slides= tiny.cc/gale15 SBSE = CPU-intensive Explosive growth of SBSE papers ai4se.net
  • 8. 8 src= tiny.cc/gale15code slides= tiny.cc/gale15 SBSE = CPU-intensive Evaluates 1000s, 1,000,000s of candidates Objectives = evaluate(decisions) Cost = Generations * (Selection + Evaluation * Generation) G * (O(N2) + E * O(1) * N) Explosive growth of SBSE papers ai4se.net
  • 9. 9 src= tiny.cc/gale15code slides= tiny.cc/gale15 This talk • What is search-based SE? • Why use less CPU for SBSE? • How to use less CPU – Refactor the optimizer: – Add in some data mining • Experimental results • Related Work • Future work • A challenge question: – Are we making this too hard? ai4se.net
  • 10. 10ai4se.net src= tiny.cc/gale15code slides= tiny.cc/gale15 • Less power – Less power generation pollution – Less barriers to usage • Less cost – of hardware of cloud time Why seek less CPU?
  • 11. 11ai4se.net src= tiny.cc/gale15code slides= tiny.cc/gale15 • Less generation of candidates – Less confusion • Verrappa and Letier: – “..for industrial problems, these algorithms generate (many) solutions (makes) understanding them and selecting one among them difficult and time consuming” https://goo.gl/LvsQd n Why seek less CPU?
  • 12. 12ai4se.net src= tiny.cc/gale15code slides= tiny.cc/gale15 When searching for solutions “you don’t need all that detail” In Theorem proving • Narrows (Amarel, 1986) • Master variables (Crawford 1995) • Back doors (Selman 2002). In Software Eng. • Saturation in mutation testing (Budd, 1980 and many others In Computer Graphics In Machine learning • Variable subset selection (Kohavi, 1997) • Instance selection (Chen, 1975) • Active learning
  • 13. 13 src= tiny.cc/gale15code slides= tiny.cc/gale15 This talk • What is search-based SE? • Why use less CPU for SBSE? • How to use less CPU – Refactor the optimizer: – Add in some data mining • Experimental results • Related Work • Future work • A challenge question: – Are we making this too hard? ai4se.net
  • 14. 14 src= tiny.cc/gale15code slides= tiny.cc/gale15 Objectives = evaluate(decisions) Generations *( Selection + Evaluation * Generation) G * ( O(N2) + E * O(1)*N ) How to use less CPU (for SBSE) ai4se.net
  • 15. 15 src= tiny.cc/gale15code slides= tiny.cc/gale15 Objectives = evaluate(decisions) Generations * (Selection + Evaluation * Generation) G * ( O(N2) + E * O(1)*N ) How to use less CPU (for SBSE) Approximate the space • k=2 divisive clustering ai4se.net
  • 16. 16 src= tiny.cc/gale15code slides= tiny.cc/gale15 Objectives = evaluate(decisions) Generations * (Selection + Evaluation * Generation) G * ( O(N2) + E * O(1)*N ) How to use less CPU (for SBSE) Approximate the space • k=2 divisive clustering (X,Y)= 2 very distant points in O(2N) ai4se.net
  • 17. 17 src= tiny.cc/gale15code slides= tiny.cc/gale15 Objectives = evaluate(decisions) Generations * (Selection + Evaluation * Generation) G * ( O(N2) + E * O(1)*N ) How to use less CPU (for SBSE) Approximate the space • k=2 divisive clustering (X,Y)= 2 very distant points in O(2N) Evaluate only (X,Y) ai4se.net
  • 18. 18 src= tiny.cc/gale15code slides= tiny.cc/gale15 Objectives = evaluate(decisions) Generations * (Selection + Evaluation * Generation) G * ( O(N2) + E * O(1)*N ) How to use less CPU (for SBSE) Approximate the space • k=2 divisive clustering (X,Y)= 2 very distant points in O(2N) Evaluate only (X,Y) If better(X,Y) • If size(cluster) > sqrt(N) – Split, recurse on better half – E.g. cull red • Else, push points towards X – E.g. push orange ai4se.net
  • 19. 19 src= tiny.cc/gale15code slides= tiny.cc/gale15 Objectives = evaluate(decisions) Generations * (Selection + Evaluation * Generation) G * ( O(N2) + E * O(1)*N ) How to use less CPU (for SBSE) Red is culled Approximate the space • k=2 divisive clustering (X,Y)= 2 very distant points in O(2N) Evaluate only (X,Y) If better(X,Y) • If size(cluster) > sqrt(N) – Split, recurse on better half – E.g. cull red • Else, push points towards X – E.g. push orange ai4se.net
  • 20. 20 src= tiny.cc/gale15code slides= tiny.cc/gale15 e.g. orange points get pushed this way Objectives = evaluate(decisions) Generations * (Selection + Evaluation * Generation) G * ( O(N2) + E * O(1)*N ) How to use less CPU (for SBSE) Red is culled Approximate the space • k=2 divisive clustering (X,Y)= 2 very distant points in O(2N) Evaluate only (X,Y) If better(X,Y) • If size(cluster) > sqrt(N) – Split, recurse on better half – E.g. cull red • Else, push points towards X – E.g. push orange ai4se.net
  • 21. 21 src= tiny.cc/gale15code slides= tiny.cc/gale15 e.g. orange points get pushed this way Objectives = evaluate(decisions) Generations * (Selection + Evaluation * Generation) G * O(N2) + E * O(1)*N How to use less CPU (for SBSE) Red is culled g * ( O(N) + log( E * O(1) * N)) Approximate the space • k=2 divisive clustering (X,Y)= 2 very distant points in O(2N) Evaluate only (X,Y) If better(X,Y) • If size(cluster) > sqrt(N) – Split, recurse on better half – E.g. cull red • Else, push points towards X – E.g. push orange ai4se.net
  • 23. 23 src= tiny.cc/gale15code slides= tiny.cc/gale15 GALE’s clustering = fast analog for PCA (so GALE is a heuristic spectral learner) 23ai4se.net
  • 24. 24 src= tiny.cc/gale15code slides= tiny.cc/gale15 This talk • What is search-based SE? • Why use less CPU for SBSE? • How to use less CPU – Refactor the optimizer: – Add in some data mining • Experimental results • Related Work • Future work • A challenge question: – Are we making this too hard? 24ai4se.net
  • 25. 25ai4se.net src= tiny.cc/gale15code slides= tiny.cc/gale15 Sample models Benchmark suites (small) • The usual suspects: goo.gl/FTyhkJ – 2-3 line equations – Fonseca, Schaffer, woBar. Golinski, • Also, from goo.gl/w98wxu – The ZDT suite : – The DTLZ suite SE models • On-line at: goo.gl/nv2AVK – XOMO: goo.gl/tY4nLu COCOMO software effort estimator + defect prediction + risk advisor – POM3: goo.gl/RMxWC Agile teams prioritizing tasks • Tasks costs and utility may subsequently change • Teams depend on products from other teams • Internal NASA models: – CDA: goo.gl/wLVrYA • NASA’s requirements models for human avionics
  • 26. 26ai4se.net src= tiny.cc/gale15code slides= tiny.cc/gale15 Comparison algorithms What we used (in paper) • NSGA-II (of course) • SPEA2 • Selected from Sayyad et al’s ICSE’13 survey of “usually used MOEAs in SE” Not IBEA: – BTW, I don’t like IBEA, just its continuous domination function – Used in GALE Since paper • Differential evolution • MOEA/D • ?NSGA III – Some quirky “bunching problems”
  • 27. 27 src= tiny.cc/gale15code slides= tiny.cc/gale15 GALE: one of the best, far fewer evals Gray: stats tests: as good as the best ai4se.net
  • 28. 28 src= tiny.cc/gale15code slides= tiny.cc/gale15 For small models, not much slower For big models, 100 times faster ai4se.net
  • 29. 29 src= tiny.cc/gale15code slides= tiny.cc/gale15 On big models, GALE does very well NASA’s requirements models for human avionics • GALE: 4 mins • NSGA-II: 8 hours ai4se.net
  • 31. 31 src= tiny.cc/gale15code slides= tiny.cc/gale15 This talk • What is search-based SE? • Why use less CPU for SBSE? • How to use less CPU – Refactor the optimizer: – Add in some data mining • Experimental results • Related Work • Future work • A challenge question: – Are we making this too hard? ai4se.net
  • 32. 32ai4se.net src= tiny.cc/gale15code slides= tiny.cc/gale15 Related work (more) • Active learning [8] – Don’t evaluate all, – Just the most interesting • Kamvar et al. 2003 [33] – Spectral learning • Boley , PDDP 1998 [34] – Classification, recursive descent on PCA component – O(N2), not O(N) • SPEA2, NSGA-II, PSO, DE, MOEA/D, Tabu.. – All O(N) evaluations • Various local search methods (Peng [40]) – None known in SE – None boasting GALE’s reduced runtimes • Response surface methods Zuluaga [8] – Parametric assumptions about Pareto frontier – Active learning [X] = reference in paper
  • 33. 33 src= tiny.cc/gale15code slides= tiny.cc/gale15 This talk • What is search-based SE? • Why use less CPU SBSE? • How to use less CPU – Refactor the optimizer: – Add in some data mining • Experimental results • Related Work • Future work • A challenge question: – Are we making this too hard? ai4se.net
  • 34. 34ai4se.net src= tiny.cc/gale15code slides= tiny.cc/gale15 Future work More Models • Siegmund & Apel’s runtime configuration models • Rungta’s NASA models of space pilots flying MARS missions • 100s of Horkoff’s softgoal models • Software product lines More Tool Building • Explanation systems – Complex MOEA tasks solved by reflecting on only a few dozen examples – Human in the loop guidance for the inference? • There remains one loophole GALE did not exploit – So after GALE comes STORM, – Work in progress
  • 35. 35 src= tiny.cc/gale15code slides= tiny.cc/gale15 This talk • What is search-based SE? • Why use less CPU for SBSE? • How to use less CPU – Refactor the optimizer: – Add in some data mining • Experimental results • Related Work • Future work • A challenge question: – Are we making this too hard? ai4se.net
  • 36.
  • 37. 37 src= tiny.cc/gale15code slides= tiny.cc/gale15 GALE’s dangerous idea • Simple approximations exist for seemingly complex problems. • Researchers jump to the complex before exploring the simpler. • Test supposedly sophisticated vs simpler alternates (the straw man). • My career: “my straw don’t burn” ai4se.net
  • 38.