Your SlideShare is downloading. ×
0
On the Value of User Preferences in Search-Based Software Engineering:
On the Value of User Preferences in Search-Based Software Engineering:
On the Value of User Preferences in Search-Based Software Engineering:
On the Value of User Preferences in Search-Based Software Engineering:
On the Value of User Preferences in Search-Based Software Engineering:
On the Value of User Preferences in Search-Based Software Engineering:
On the Value of User Preferences in Search-Based Software Engineering:
On the Value of User Preferences in Search-Based Software Engineering:
On the Value of User Preferences in Search-Based Software Engineering:
On the Value of User Preferences in Search-Based Software Engineering:
On the Value of User Preferences in Search-Based Software Engineering:
On the Value of User Preferences in Search-Based Software Engineering:
On the Value of User Preferences in Search-Based Software Engineering:
On the Value of User Preferences in Search-Based Software Engineering:
On the Value of User Preferences in Search-Based Software Engineering:
On the Value of User Preferences in Search-Based Software Engineering:
On the Value of User Preferences in Search-Based Software Engineering:
On the Value of User Preferences in Search-Based Software Engineering:
On the Value of User Preferences in Search-Based Software Engineering:
On the Value of User Preferences in Search-Based Software Engineering:
On the Value of User Preferences in Search-Based Software Engineering:
On the Value of User Preferences in Search-Based Software Engineering:
On the Value of User Preferences in Search-Based Software Engineering:
On the Value of User Preferences in Search-Based Software Engineering:
On the Value of User Preferences in Search-Based Software Engineering:
On the Value of User Preferences in Search-Based Software Engineering:
On the Value of User Preferences in Search-Based Software Engineering:
On the Value of User Preferences in Search-Based Software Engineering:
On the Value of User Preferences in Search-Based Software Engineering:
On the Value of User Preferences in Search-Based Software Engineering:
On the Value of User Preferences in Search-Based Software Engineering:
On the Value of User Preferences in Search-Based Software Engineering:
On the Value of User Preferences in Search-Based Software Engineering:
On the Value of User Preferences in Search-Based Software Engineering:
On the Value of User Preferences in Search-Based Software Engineering:
On the Value of User Preferences in Search-Based Software Engineering:
On the Value of User Preferences in Search-Based Software Engineering:
On the Value of User Preferences in Search-Based Software Engineering:
On the Value of User Preferences in Search-Based Software Engineering:
On the Value of User Preferences in Search-Based Software Engineering:
On the Value of User Preferences in Search-Based Software Engineering:
On the Value of User Preferences in Search-Based Software Engineering:
On the Value of User Preferences in Search-Based Software Engineering:
On the Value of User Preferences in Search-Based Software Engineering:
Upcoming SlideShare
Loading in...5
×

Thanks for flagging this SlideShare!

Oops! An error has occurred.

×
Saving this for later? Get the SlideShare app to save on your phone or tablet. Read anywhere, anytime – even offline.
Text the download link to your phone
Standard text messaging rates apply

On the Value of User Preferences in Search-Based Software Engineering:

587

Published on

Tim Menzies, …

Tim Menzies,
Abdel Salam Sayyad,
Hany Ammar

Published in: Education
0 Comments
1 Like
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total Views
587
On Slideshare
0
From Embeds
0
Number of Embeds
0
Actions
Shares
0
Downloads
8
Comments
0
Likes
1
Embeds 0
No embeds

Report content
Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
No notes for slide

Transcript

  • 1. On#the#Value#of#User#Preferences#in# Search4Based#So7ware#Engineering:##A#Case#Study#in#So7ware#Product#Lines## # Tim#Menzies,## Abdel#Salam#Sayyad,# Hany#Ammar# # WVU,#USA# Nov’12## #
  • 2. Sound#bites#•  The#new#age#of#the#app# #•  Stop#Nnkering#with#small#stuff# #•  Enough#with#the#usual#suspects:## –  NSGA4II,#SPEA2,#etc# #•  If#preferences#maSer# –  Then#the##best#opNmizer#understands#preferences#the#best# 2#
  • 3. Roadmap# ①  Feature(based,SE, ②  Algorithms, ③  IBEA, ④  Tree,muta<on, ⑤  Conclusion, #
  • 4. Roadmap# ①  Feature(based,SE, ②  Algorithms, ③  IBEA, ④  Tree,muta<on, ⑤  Conclusion, #
  • 5. WELCOME,TO,,THE,NEW,WORLDE,,, 5#
  • 6. The#Nmes,#they#are#a#changing# Olde,worlde:,, New,worlde:,, product(based,SE, app(based,SE,,e.g.#Microso7#office# •  E.g.#Apple#app#store# 6#
  • 7. The#Nmes,#they#are#a#changing# Olde,worlde:,, New,worlde:,, product(based,SE, app(based,SE,,•  Vendors#tried#to#retain#their# •  Smart#phones#and#tablet4 user#base#via#some# based#so7ware# complete#ecologies# # # •  Users#choosing#many# numbers#of#small#apps#from#•  One#so7ware#soluNon#for# different#vendors,## all#user#needs###(e.g.# –  each#performing#a#specific# Microso7#Office).## small#task.# # #•  Large,#complex,#so7ware# •  Vendors#must#quickly#and# plaorms,## conNnually#reconfigure#apps## –  To#retain#and#extend#their# –  very#slow#to#change.## customer#base.# 7#
  • 8. Feature–oriented#domain#analysis#•  Feature#maps#=#a# lightweight#method#for# defining#a#space#of##opNons#•  Product4line#configuraNons#•  Defacto#standard#for# modeling#variability## 8#
  • 9. Original#FODA#paper#:#2700+#citaNons# Half#since# 2007# 9#
  • 10. hSp://www.splot4research.org/# 200+#models,##plus#an#instance#generator# 10#
  • 11. LINUX#kernel#=#6000+#features#86%#declare#constraints#of#some#sort,##Most#features#refer#to#244#other#features.### 11#
  • 12. Need#for#beSer#automaNon#•  Such#complexity#needs#automated#support# –  especially##feature#models##combined#with# •  #user#preferences#and#prioriNes,#e.g.#cost#and#reliability.## Search#for#valid#products:## 9#state#of#the#art#theorem# provers##[Pohl,#ASE’11]# # Bad## scalability# And#these#were# “simple”#models# 12#
  • 13. Diving##deeper#•  Much#prior#work#explored## Nny#objecNve#spaces## –  Two#or#three#objecNves# –  Or,#higher#(but#only#for#small#models)# # # #•  So7ware#engineering#=#navigaNng#compeNng#concerns# 1.  That#saNsfies#most#domain#constraints#(0#≤###violaNons#≤#100%)# The# usual# 2.  That#offers#most#features# suspects# 3.  Build#“stuff”#In#least#Nme# 4.  That#we#have#used#most#before# 13# 5.  Using#features#with#least#known#defects#
  • 14. Roadmap# ①  Feature(based,SE, ②  Algorithms, ③  IBEA, ④  Tree,muta<on, ⑤  Conclusion, #
  • 15. MOEA=#MulN4objecNve### evoluNonary#algorithms#######################•  Repeat#Nll#happy#or#exhausted# –  SelecNon#(cull#the#herd)# –  Cross4over#(the#rude#bit)# –  MutaNon#(stochasNc#jiggle)# 15#
  • 16. Some#MOEA#ApplicaNon#Domain Application TypesControl gas pipeline, pole balancing, missile evasion, pursuitDesign semiconductor layout, aircraft design, keyboard configuration, communication networksScheduling manufacturing, facility scheduling, resource allocationRobotics trajectory planningMachine Learning designing neural networks, improving classification algorithms, classifier systemsSignal Processing filter designGame Playing poker, checkers, prisoner’s dilemmaCombinatorial set covering, travelling salesman, routing, bin packing,Optimization graph colouring and partitioning
  • 17. MOEA#for#Search4based#SE#TransformaNon #Cooper,#Ryan,#Schielke,#Subramanian,#FaNregun,#Williams#Requirements## #Bagnall,#Mansouri,#Zhang#Effort#predicNon# #Aguilar4Ruiz,#Burgess,#Dolado,#Lefley,#Shepperd##Management # #Alba,#Antoniol,#Chicano,#Di#Pentam#Greer,#Ruhe#Heap#allocaNon #Cohen,#Kooi,#Srisa4an##Regression#test #Li,#Yoo,#Elbaum,#Rothermel,#WalcoS,#Soffa,#Kampxamer##SOA## # # #Canfora,#Di#Penta,#Esposito,#Villani##Refactoring # #Antoniol,#Briand,#Cinneide,#O’Keeffe,#Merlo,#Seng,#TraS#Test#GeneraNon #Alba,#Binkley,#BoSaci,#Briand,#Chicano,#Clark,#Cohen,#Gutjahr,## ## # # #Harrold,#Holcombe,#Jones,#Korel,#Pargass,#Reformat,#Roper,#McMinn,# ## # # #Michael,#Sthamer,#Tracy,#Tonella,Xanthakis,#Xiao,#Wegener,#Wilkins#Maintenance # #Antoniol,#Lutz,#Di#Penta,#Madhavi,#Mancoridis,#Mitchell,#Swi7#Model#checking #Alba,#Chicano,#Godefroid#Probing # # #Cohen,#Elbaum##UIOs# # # #Derderian,#Guo,#Hierons# So#study#FODA4#Comprehension #Gold,#Li,#Mahdavi# to#learn#how#to#Protocols# # #Alba,#Clark,#Jacob,#Troya# improve#these#Component#sel #Baker,#SkalioNs,#Steinhofel,#Yoo# tasks.#Agent#Oriented #Haas,#Peysakhov,#Sinclair,#Shami,#Mancoridis# 17#
  • 18. Much#increased#interest## in#Search4based#SE# 18#
  • 19. The#Pareto#FronNer#•  Mutants#=#<D,O>#=#<decisions,#objecNves># –  E.g.#car# •  Decisions:#color#of#car,#number#of#cylinders,#number#of#wheels# •  ObjecNves:#miles#per#hour,#cost##(objecNves#may#complete)# –  E.g.#learning#formula# •  Decisions:#what#variables#and#constants#to#use# •  ObjecNves:#model#simplicity#vs#effecNveness#(objecNves#may# complete)#•  Pareto#fronNer:#select#the#non4dominated#mutants# –  X#dominates#Y## •  if#for#all#objecNves,##X#is#never#any#worse##than#Y# •  If#for#one#objecNve,#X#beSer#than#Y# 19#
  • 20. Once#you#know#fronNer# •#Select#from#here#Issues:# •#Ignore#here#1)  Spread#2)  Hypervolume#3)  ComputaNonal#cost:##“g”##generaNons,#M#mutants,#O(gM2)# 20#
  • 21. The#usual#suspects:# ##=#NSGA4II### #=#SPEA2# In#this#case,#NSGA4II# gets#more#spread# Combines#N#objecNves## to#one##with#some## weighNng#scheme# 21#
  • 22. Some#details#on#the#usual#suspects#NSGA(II,, SPEA2,,•  Is#a#geneNc#algorithm# •  Is#a#geneNc#algorithm# # # •  Changes#the#definiNon#of#•  Non4dominated#sort# “dominaNon”# –  HeurisNc#way#to#fast#group# –  SPEA#(version#1)#scored#mutants#by# mutants#into#bands# how#many#others#they#dominated# # –  Got#confused#by#overlaps#in#the# dominaNon#sets#•  Crowd#pruning##via#approximate# # hypercube#around#each#mutant:# •  SPEA#(version#2):# O(Onlogn)# –  Adds#a#“local#density#factor”#to#the# dominaNon#weight# 3# –  Mutants#in#dense#areas#valued## 2# more# ## 1# •  SPEA2#beSer#than#SPEA1# 2# 22#
  • 23. Any#number#of#opNmizaNons## to#tradiNonal#GAs#•  The#history#of#MOEAs#in#the#last# •  DifferenNal#evoluNon#(Storn#1996)# 15#years#is# –  Mutate#by#interpolaNons#between# –  OpNmize#via#hybrid#GA#+#other# exisNng#mutants# search#method# # –  For#x#in#mutants## ####y#=#any1#+#extrapolate(any3#–#any2)#•  Local#search:## ####if#y#dominates#x#then#x#=#y# –  before#select,#do#a#liSle#simulated# # annealing#on#X%#of#the#populaNon# # •  Cellular#automata,#•  ScaSer#search#(Glover’s#next# •  #Ant#colony#opNmizaNon,#### generaNon#tabu#search)# –  Includes#a#liSle#local#search# •  Bayesian#staNsNcs#to#bias#the# # mutaNon,##•  ParNcle#swam#opNmizaNon# •  Etc## –  May#do#as#well#as#scaSer#search##(Yin# •  etc# and#Glover#2007)# 23#
  • 24. Roadmap# ①  Feature(based,SE, ②  Algorithms, ③  IBEA, ④  Tree,muta<on, ⑤  Conclusion, #
  • 25. Three#groups#of#Algorithms#DominaNon# DominaNon#Is#a#binary# PSO# Is#a#conNnuous#concept# concept# DE# ScaSer## IBEA# Spea2# search# Aggressive# exploraNon# Indicator4based## of#preference## SA# evoluNonary# space# Nsga4II# algorithms# mocell# Z3# SMT#solvers# 25#
  • 26. IBEA#•  Bo#smarts#anywhere#except#in#the#exploraNon#of#preferences#•  I(x1,x2):# –  Least#adjust#objecNve#scores#such#that#x1#dominates#x2#•  Repeat#Nll#just#a#few#le7# –  Score#each#instance#x1##buy#summing#its#“I”#to#everyone#else# # K=# # 0.05# # # # # –  Sort#all#instances#by#F# –  Delete#worst#•  Then,#standard#GA#(cross4over,#mutaNon)#on#the#survivors# ## 26#
  • 27. Case#studies#Data#from#hSp://www.splot4research.org/#Algorithms#from#jMetal:#hSp://jmetal.sourceforge.net/## Cross4tree# constraints# 27#
  • 28. 4#studies:# Bi,#tri,#quad,#five4#objecNves#So7ware#engineering#=#navigaNng#compeNng#concerns# 1.  That#saNsfies#most#domain#constraints#(0#≤###violaNons#≤# 100%)# 2.  That#offers#most#features# 3.  Build#“stuff”#In#least#Nme# 4.  That#we#have#used#most#before# 5.  Using#features#with#least## known#defects# # # Binary#objecNves#=#1,2# Tri4objecNve#########=#1,2,3# Quad4objecNve####=#1,2,3,4# Five4objecNve######=#1,2,3,4,5# 28#
  • 29. #HV#############=#hypervolume#of#dominated#region#Spread######=#coverage#of#fronNer#%#correct#=#%constraints#saNsfied# 29#
  • 30. #HV#############=#hypervolume#of#dominated#region#Spread######=#coverage#of#fronNer#%#correct#=#%constraints#saNsfied#Comment,1:,all#about#the#same#for#the#24objecNve#problem# 30#
  • 31. #HV#############=#hypervolume#of#dominated#region#Spread######=#coverage#of#fronNer#%#correct#=#%constraints#saNsfied#Comment,2:,E4shop#is#a#nasty#problem:#needs#50M#evals# 31#
  • 32. #HV#############=#hypervolume#of#dominated#region#Spread######=#coverage#of#fronNer#%#correct#=#%constraints#saNsfied#Comment,3:,IBEA#has#no#spread#operators,#but#gets#best#spread#32#
  • 33. #HV#############=#hypervolume#of#dominated#region#Spread######=#coverage#of#fronNer#%#correct#=#%constraints#saNsfied#Comment,4:,IBEA#has#no#HV#operators,#but#usually#gets#best#HV#33#
  • 34. #HV#############=#hypervolume#of#dominated#region#Spread######=#coverage#of#fronNer#%#correct#=#%constraints#saNsfied#Comment,5:,All#the#non4IBEA#algorithms#are#very#similar# 34#
  • 35. #HV#############=#hypervolume#of#dominated#region#Spread######=#coverage#of#fronNer#%#correct#=#%constraints#saNsfied#Comment,6:,IBEA#does#much,#much##beSer#on#constraints# 35#
  • 36. Why#is#this#interesNng?#Other,MOEAs, IBEA,•  The#usual#suspects#are#widely,# •  Rather#stupid#on#those# uncriNcally#used#in#many#MOEA# applicaNons# internal#tricks# –  E.g.#especially#NSGA4II#and#SPEA2# –  Just#does#a#ye#olde#crossover#•  Focused#on#internal#algorithmic# mutate#GA# tricks# –  Plus:#aggressive#exploraNon# –  Techniques#for# of#the#preference#space# •  #improving#spread## •  Improving#HV# •  And#the#net#effect#of#all# •  Avoid#overlaps#in#cross4over#of# dominated#space# those#differences# •  etc# –  BeSer#spreads#•  And#the#net#effect#of#all#those# –  BeSer#HV# differences?# –  Not#much# –  Fewer#constraint#violaNons# Conclusion:# preference#is#power# 36#
  • 37. Roadmap# ①  Feature(based,SE, ②  Algorithms, ③  IBEA, ④  Tree,muta<on, ⑤  Conclusion, #
  • 38. What#about#non4MOEA#soluNons?#DominaNon# DominaNon#Is#a#binary# PSO# Is#a#conNnuous#concept# concept# DE# ScaSer## IBEA# Spea2# search# Aggressive# exploraNon# Indicator4based## of#preference## SA# evoluNonary# space# Nsga4II# algorithms# mocell# Z3# SMT#solvers# 38#
  • 39. Ethan’s#complaint#•  So7ware#engineer## designs#are#o7en## nested#hierarchical## constraints#•  Ethan#Jackson,#Microso7,## advocate#for#the#Z3#SMT#solver:# –  Why#mutate#at#random,#then#check#for# constraint?# –  BeSer#to#drive#the#mutaNons#by#the# constraints?# 39#
  • 40. Dump#MOEAs?## Move#to#more#logical#forms?#Pro:, Con:,move,to,,say,,SMT,solvers, stay,with,MOEA,•  Next#generaNon#of# •  ImplementaNon#complexity# algorithm# less# –  The#next#big#thing# •  More#tools#•  BeSer#to#drive#the# •  Easier#modificaNon#and# mutaNons#by#the# experimentaNon# constraints?# •  Models#can#be#expressed# anyway#you#like# 40#
  • 41. Tree#mutaNon#•  Work#in#progress#•  Simple#adaptaNon#of#current#MOEAs#for#systems# of#hierarchical#constraints# #if#rand(0,1)<#mutaNon_probability:############Dont#mutate#if#youre#violaNng#one#of#the#rules:# # #1)#if#deselecNng#root#feature# # #2)#if#selecNng#feature#whose############################parent#is#not#selected# # #3)#if#deselecNng#feature#that##########################another#selected#feature#requires# # #4)#if#group#cardinality#violaNon# #else:# # #flip#this#bit# # # ##### # # #if#selecNng#(turning#on)#a#feature#then# # # # #turn#on#children### IBEA# # # #else#if#deselecNng#(turning#off)# stabilizes#70########################################feature#then:# Nmes#faster# # # ######### #########turn#off#all#children# 41#
  • 42. Tree#mutaNon#preserves## domain#constraints# So#what## case#for# SMT?# 42#
  • 43. Roadmap# ①  Feature(based,SE, ②  Algorithms, ③  IBEA, ④  Tree,muta<on, ⑤  Conclusion, #
  • 44. Sound#bites#•  The#new#age#of#the#app# –  In,this,new,worlde:,,use,FODA, (feature(oriented,domain,analysis),•  Stop#Nnkering#with#small#stuff## –  Many,MOEAs,have,strikingly,, similar,performance,•  Enough#with#the#usual#suspects:## –  NSGA4II,#SPEA2,#etc# –  Too,much,uncri<cal,applica<on,of,these,algorithms,•  If#preferences#maSer# –  Then#the##best#opNmizer#understands##preferences#the#best# –  IBEA:,aggressive,preference,explora<on, –  Tree,muta<on:,respect,your,domain, 44#

×