SlideShare a Scribd company logo
On#the#Value#of#User#Preferences#in#
 Search4Based#So7ware#Engineering:##
A#Case#Study#in#So7ware#Product#Lines##
                  #
              Tim#Menzies,##
           Abdel#Salam#Sayyad,#
              Hany#Ammar#
                    #
               WVU,#USA#
                 Nov’12##
                    #
Sound#bites#
•  The#new#age#of#the#app#
   #
•  Stop#Nnkering#with#small#stuff#
   #
•  Enough#with#the#usual#suspects:##
   –  NSGA4II,#SPEA2,#etc#
      #
•  If#preferences#maSer#
   –  Then#the##best#opNmizer#understands#preferences#the#best#




                                                                  2#
Roadmap#




  ①    Feature(based,SE,
  ②    Algorithms,
  ③    IBEA,
  ④    Tree,muta<on,
  ⑤    Conclusion,
  #
Roadmap#




  ①    Feature(based,SE,
  ②    Algorithms,
  ③    IBEA,
  ④    Tree,muta<on,
  ⑤    Conclusion,
  #
WELCOME,TO,,
THE,NEW,WORLDE,,
,




                   5#
The#Nmes,#they#are#a#changing#
      Olde,worlde:,,             New,worlde:,,
    product(based,SE,            app(based,SE,,
e.g.#Microso7#office#     •  E.g.#Apple#app#store#




                                                   6#
The#Nmes,#they#are#a#changing#
         Olde,worlde:,,                     New,worlde:,,
       product(based,SE,                    app(based,SE,,
•  Vendors#tried#to#retain#their# •  Smart#phones#and#tablet4
   user#base#via#some#               based#so7ware#
   complete#ecologies#               #
   #                              •  Users#choosing#many#
                                     numbers#of#small#apps#from#
•  One#so7ware#soluNon#for#          different#vendors,##
   all#user#needs###(e.g.#             –  each#performing#a#specific#
   Microso7#Office).##                      small#task.#
   #                                      #
•  Large,#complex,#so7ware#       •  Vendors#must#quickly#and#
   plaorms,##                       conNnually#reconfigure#apps##
                                      –  To#retain#and#extend#their#
   –  very#slow#to#change.##             customer#base.#
                                                                       7#
Feature–oriented#domain#analysis#
•  Feature#maps#=#a#
   lightweight#method#for#
   defining#a#space#of##opNons#
•  Product4line#configuraNons#
•  Defacto#standard#for#
   modeling#variability##




                                 8#
Original#FODA#paper#:#2700+#citaNons#




                              Half#since#
                                2007#


                                            9#
hSp://www.splot4research.org/#




      200+#models,##plus#an#instance#generator#   10#
LINUX#kernel#=#6000+#features#
86%#declare#constraints#of#some#sort,##
Most#features#refer#to#244#other#features.##
#




                                               11#
Need#for#beSer#automaNon#
•  Such#complexity#needs#automated#support#
  –  especially##feature#models##combined#with#
     •  #user#preferences#and#prioriNes,#e.g.#cost#and#reliability.##

                                                        Search#for#valid#products:##
                                                        9#state#of#the#art#theorem#
                                                        provers##[Pohl,#ASE’11]#
                                                        #

                                                                      Bad##
                                                                   scalability#



                                                         And#these#were#
                                                         “simple”#models#
                                                                                  12#
Diving##deeper#
•  Much#prior#work#explored##
   Nny#objecNve#spaces##
    –  Two#or#three#objecNves#
    –  Or,#higher#(but#only#for#small#models)#
       #
       #




    #
•  So7ware#engineering#=#navigaNng#compeNng#concerns#
    1.    That#saNsfies#most#domain#constraints#(0#≤###violaNons#≤#100%)#      The#
                                                                             usual#
    2.    That#offers#most#features#                                        suspects#
    3.    Build#“stuff”#In#least#Nme#
    4.    That#we#have#used#most#before#
                                                                                 13#
    5.    Using#features#with#least#known#defects#
Roadmap#




  ①    Feature(based,SE,
  ②    Algorithms,
  ③    IBEA,
  ④    Tree,muta<on,
  ⑤    Conclusion,
  #
MOEA=#MulN4objecNve###
             evoluNonary#algorithms#######################
•  Repeat#Nll#happy#or#exhausted#
   –  SelecNon#(cull#the#herd)#
   –  Cross4over#(the#rude#bit)#
   –  MutaNon#(stochasNc#jiggle)#




                                                  15#
Some#MOEA#ApplicaNon#
Domain              Application Types
Control             gas pipeline, pole balancing, missile evasion, pursuit

Design              semiconductor layout, aircraft design, keyboard
                    configuration, communication networks
Scheduling          manufacturing, facility scheduling, resource allocation

Robotics            trajectory planning

Machine Learning    designing neural networks, improving classification
                    algorithms, classifier systems
Signal Processing   filter design

Game Playing        poker, checkers, prisoner’s dilemma

Combinatorial       set covering, travelling salesman, routing, bin packing,
Optimization        graph colouring and partitioning
MOEA#for#Search4based#SE#
TransformaNon      #Cooper,#Ryan,#Schielke,#Subramanian,#FaNregun,#Williams#
Requirements##     #Bagnall,#Mansouri,#Zhang#
Effort#predicNon#   #Aguilar4Ruiz,#Burgess,#Dolado,#Lefley,#Shepperd##
Management #       #Alba,#Antoniol,#Chicano,#Di#Pentam#Greer,#Ruhe#
Heap#allocaNon     #Cohen,#Kooi,#Srisa4an##
Regression#test    #Li,#Yoo,#Elbaum,#Rothermel,#WalcoS,#Soffa,#Kampxamer##
SOA##    #    #    #Canfora,#Di#Penta,#Esposito,#Villani##
Refactoring #      #Antoniol,#Briand,#Cinneide,#O’Keeffe,#Merlo,#Seng,#TraS#
Test#GeneraNon     #Alba,#Binkley,#BoSaci,#Briand,#Chicano,#Clark,#Cohen,#Gutjahr,##
   ##    #    #    #Harrold,#Holcombe,#Jones,#Korel,#Pargass,#Reformat,#Roper,#McMinn,#
   ##    #    #    #Michael,#Sthamer,#Tracy,#Tonella,Xanthakis,#Xiao,#Wegener,#Wilkins#
Maintenance #      #Antoniol,#Lutz,#Di#Penta,#Madhavi,#Mancoridis,#Mitchell,#Swi7#
Model#checking     #Alba,#Chicano,#Godefroid#
Probing #     #    #Cohen,#Elbaum##
UIOs#    #    #    #Derderian,#Guo,#Hierons#                         So#study#FODA4#
Comprehension      #Gold,#Li,#Mahdavi#                               to#learn#how#to#
Protocols#    #    #Alba,#Clark,#Jacob,#Troya#                        improve#these#
Component#sel      #Baker,#SkalioNs,#Steinhofel,#Yoo#                     tasks.#
Agent#Oriented     #Haas,#Peysakhov,#Sinclair,#Shami,#Mancoridis#

                                                                                   17#
Much#increased#interest##
  in#Search4based#SE#




                            18#
The#Pareto#FronNer#
•  Mutants#=#<D,O>#=#<decisions,#objecNves>#
   –  E.g.#car#
        •  Decisions:#color#of#car,#number#of#cylinders,#number#of#wheels#
        •  ObjecNves:#miles#per#hour,#cost##(objecNves#may#complete)#
   –  E.g.#learning#formula#
        •  Decisions:#what#variables#and#constants#to#use#
        •  ObjecNves:#model#simplicity#vs#effecNveness#(objecNves#may#
           complete)#

•  Pareto#fronNer:#select#the#non4dominated#mutants#
   –  X#dominates#Y##
        •  if#for#all#objecNves,##X#is#never#any#worse##than#Y#
        •  If#for#one#objecNve,#X#beSer#than#Y#

                                                                             19#
Once#you#know#fronNer#
                                                       •#Select#from#here#
Issues:#                                               •#Ignore#here#
1)  Spread#
2)  Hypervolume#
3)  ComputaNonal#cost:##“g”##generaNons,#M#mutants,#O(gM2)#                 20#
The#usual#suspects:#
   ##=#NSGA4II###
    #=#SPEA2#
                        In#this#case,#NSGA4II#
                        gets#more#spread#




                       Combines#N#objecNves##
                       to#one##with#some##
                       weighNng#scheme#




                                         21#
Some#details#on#the#usual#suspects#
NSGA(II,,                            SPEA2,,
•  Is#a#geneNc#algorithm#            •  Is#a#geneNc#algorithm#
   #                                    #
                                     •  Changes#the#definiNon#of#
•  Non4dominated#sort#                  “dominaNon”#
    –  HeurisNc#way#to#fast#group#       –  SPEA#(version#1)#scored#mutants#by#
       mutants#into#bands#                  how#many#others#they#dominated#
       #                                 –  Got#confused#by#overlaps#in#the#
                                            dominaNon#sets#
•  Crowd#pruning##via#approximate#          #
   hypercube#around#each#mutant:#    •  SPEA#(version#2):#
   O(Onlogn)#                            –  Adds#a#“local#density#factor”#to#the#
                                            dominaNon#weight#
                                3#       –  Mutants#in#dense#areas#valued##
                           2#               more#
                                            ##
                      1#             •  SPEA2#beSer#than#SPEA1#
       2#                                                                      22#
Any#number#of#opNmizaNons##
                  to#tradiNonal#GAs#
•  The#history#of#MOEAs#in#the#last#            •  DifferenNal#evoluNon#(Storn#1996)#
   15#years#is#                                     –  Mutate#by#interpolaNons#between#
    –  OpNmize#via#hybrid#GA#+#other#                  exisNng#mutants#
       search#method#
       #                                            –  For#x#in#mutants##
                                                       ####y#=#any1#+#extrapolate(any3#–#any2)#
•  Local#search:##                                     ####if#y#dominates#x#then#x#=#y#
    –  before#select,#do#a#liSle#simulated#         #
       annealing#on#X%#of#the#populaNon#
       #                                        •  Cellular#automata,#
•  ScaSer#search#(Glover’s#next#                •  #Ant#colony#opNmizaNon,####
   generaNon#tabu#search)#
    –  Includes#a#liSle#local#search#
                                                •  Bayesian#staNsNcs#to#bias#the#
       #                                           mutaNon,##
•  ParNcle#swam#opNmizaNon#                     •  Etc##
    –  May#do#as#well#as#scaSer#search##(Yin#   •  etc#
       and#Glover#2007)#

                                                                                              23#
Roadmap#




  ①    Feature(based,SE,
  ②    Algorithms,
  ③    IBEA,
  ④    Tree,muta<on,
  ⑤    Conclusion,
  #
Three#groups#of#Algorithms#
DominaNon#                                                        DominaNon#
Is#a#binary#
                          PSO#                                    Is#a#conNnuous#
concept#                                                          concept#
                DE#      ScaSer##             IBEA#
          Spea2#         search#                                      Aggressive#
                                                                      exploraNon#
                                              Indicator4based##       of#preference##
                         SA#                  evoluNonary#            space#
          Nsga4II#                            algorithms#

               mocell#
                                   Z3#
                               SMT#solvers#


                                                                               25#
IBEA#
•  Bo#smarts#anywhere#except#in#the#exploraNon#of#preferences#
•  I(x1,x2):#
   –  Least#adjust#objecNve#scores#such#that#x1#dominates#x2#
•  Repeat#Nll#just#a#few#le7#
   –  Score#each#instance#x1##buy#summing#its#“I”#to#everyone#else#
      #                                                                K=#
      #                                                               0.05#
      #
      #
      #
      #
   –  Sort#all#instances#by#F#
   –  Delete#worst#
•  Then,#standard#GA#(cross4over,#mutaNon)#on#the#survivors#
   ##
                                                                          26#
Case#studies#
Data#from#hSp://www.splot4research.org/#
Algorithms#from#jMetal:#hSp://jmetal.sourceforge.net/##




                              Cross4tree#
                              constraints#




                                                          27#
4#studies:#
           Bi,#tri,#quad,#five4#objecNves#
So7ware#engineering#=#navigaNng#compeNng#concerns#
   1.  That#saNsfies#most#domain#constraints#(0#≤###violaNons#≤#
        100%)#
   2.  That#offers#most#features#
   3.  Build#“stuff”#In#least#Nme#
   4.  That#we#have#used#most#before#
   5.  Using#features#with#least##
        known#defects#
   #
   #
   Binary#objecNves#=#1,2#
   Tri4objecNve#########=#1,2,3#
   Quad4objecNve####=#1,2,3,4#
   Five4objecNve######=#1,2,3,4,5#
                                                                  28#
#
HV#############=#hypervolume#of#dominated#region#
Spread######=#coverage#of#fronNer#
%#correct#=#%constraints#saNsfied#




                                                    29#
#
HV#############=#hypervolume#of#dominated#region#
Spread######=#coverage#of#fronNer#
%#correct#=#%constraints#saNsfied#




Comment,1:,all#about#the#same#for#the#24objecNve#problem#   30#
#
HV#############=#hypervolume#of#dominated#region#
Spread######=#coverage#of#fronNer#
%#correct#=#%constraints#saNsfied#




Comment,2:,E4shop#is#a#nasty#problem:#needs#50M#evals#   31#
#
HV#############=#hypervolume#of#dominated#region#
Spread######=#coverage#of#fronNer#
%#correct#=#%constraints#saNsfied#




Comment,3:,IBEA#has#no#spread#operators,#but#gets#best#spread#32#
#
HV#############=#hypervolume#of#dominated#region#
Spread######=#coverage#of#fronNer#
%#correct#=#%constraints#saNsfied#




Comment,4:,IBEA#has#no#HV#operators,#but#usually#gets#best#HV#33#
#
HV#############=#hypervolume#of#dominated#region#
Spread######=#coverage#of#fronNer#
%#correct#=#%constraints#saNsfied#




Comment,5:,All#the#non4IBEA#algorithms#are#very#similar#   34#
#
HV#############=#hypervolume#of#dominated#region#
Spread######=#coverage#of#fronNer#
%#correct#=#%constraints#saNsfied#




Comment,6:,IBEA#does#much,#much##beSer#on#constraints#   35#
Why#is#this#interesNng?#
Other,MOEAs,                                   IBEA,
•  The#usual#suspects#are#widely,#             •  Rather#stupid#on#those#
   uncriNcally#used#in#many#MOEA#
   applicaNons#                                   internal#tricks#
    –  E.g.#especially#NSGA4II#and#SPEA2#         –  Just#does#a#ye#olde#crossover#
•  Focused#on#internal#algorithmic#                  mutate#GA#
   tricks#                                        –  Plus:#aggressive#exploraNon#
    –  Techniques#for#                               of#the#preference#space#
         •  #improving#spread##
         •  Improving#HV#                      •  And#the#net#effect#of#all#
         •  Avoid#overlaps#in#cross4over#of#
            dominated#space#                      those#differences#
         •  etc#                                  –  BeSer#spreads#
•  And#the#net#effect#of#all#those#                –  BeSer#HV#
   differences?#
    –  Not#much#                                  –  Fewer#constraint#violaNons#

                                                           Conclusion:#
                                                       preference#is#power#        36#
Roadmap#




  ①    Feature(based,SE,
  ②    Algorithms,
  ③    IBEA,
  ④    Tree,muta<on,
  ⑤    Conclusion,
  #
What#about#non4MOEA#soluNons?#
DominaNon#                                                        DominaNon#
Is#a#binary#
                          PSO#                                    Is#a#conNnuous#
concept#                                                          concept#
                DE#      ScaSer##             IBEA#
          Spea2#         search#                                      Aggressive#
                                                                      exploraNon#
                                              Indicator4based##       of#preference##
                         SA#                  evoluNonary#            space#
          Nsga4II#                            algorithms#

               mocell#
                                   Z3#
                               SMT#solvers#


                                                                               38#
Ethan’s#complaint#
•  So7ware#engineer##
   designs#are#o7en##
   nested#hierarchical##
   constraints#
•  Ethan#Jackson,#Microso7,##
   advocate#for#the#Z3#SMT#solver:#
   –  Why#mutate#at#random,#then#check#for#
      constraint?#
   –  BeSer#to#drive#the#mutaNons#by#the#
      constraints?#

                                              39#
Dump#MOEAs?##
        Move#to#more#logical#forms?#
Pro:,                        Con:,
move,to,,say,,SMT,solvers,   stay,with,MOEA,
•  Next#generaNon#of#        •  ImplementaNon#complexity#
   algorithm#                   less#
   –  The#next#big#thing#    •  More#tools#
•  BeSer#to#drive#the#       •  Easier#modificaNon#and#
   mutaNons#by#the#             experimentaNon#
   constraints?#             •  Models#can#be#expressed#
                                anyway#you#like#




                                                        40#
Tree#mutaNon#
•      Work#in#progress#
•      Simple#adaptaNon#of#current#MOEAs#for#systems#
       of#hierarchical#constraints#
       #
if#rand(0,1)<#mutaNon_probability:###
#########Don't#mutate#if#you're#violaNng#one#of#the#rules:#
         #         #1)#if#deselecNng#root#feature#
         #         #2)#if#selecNng#feature#whose##
##########################parent#is#not#selected#
         #         #3)#if#deselecNng#feature#that#
#########################another#selected#feature#requires#
         #         #4)#if#group#cardinality#violaNon#
         #else:#
         #         #flip#this#bit#                #    #      #####
         #         #         #if#selecNng#(turning#on)#a#feature#then#
         #         #         #         #turn#on#children###                  IBEA#
         #         #         #else#if#deselecNng#(turning#off)#           stabilizes#70#
#######################################feature#then:#                    Nmes#faster#
         #         #         ######### #########turn#off#all#children#                     41#
Tree#mutaNon#preserves##
  domain#constraints#




          So#what##
          case#for#
           SMT?#
                           42#
Roadmap#




  ①    Feature(based,SE,
  ②    Algorithms,
  ③    IBEA,
  ④    Tree,muta<on,
  ⑤    Conclusion,
  #
Sound#bites#
•  The#new#age#of#the#app#
   –  In,this,new,worlde:,,use,FODA,
      (feature(oriented,domain,analysis),
•  Stop#Nnkering#with#small#stuff##
   –  Many,MOEAs,have,strikingly,,
      similar,performance,
•  Enough#with#the#usual#suspects:##
   –  NSGA4II,#SPEA2,#etc#
   –  Too,much,uncri<cal,applica<on,of,these,algorithms,
•  If#preferences#maSer#
   –  Then#the##best#opNmizer#understands##preferences#the#best#
   –  IBEA:,aggressive,preference,explora<on,
   –  Tree,muta<on:,respect,your,domain,
                                                                   44#

More Related Content

What's hot

Sched lgullll
Sched lgullllSched lgullll
Sched lgullll
Pablo Guzmán
 
Viva
VivaViva
Eula
EulaEula
Slow
SlowSlow
Sysinfo
SysinfoSysinfo
Sysinfo
Nadia Elise
 
Wpi log 2013.06.08_16.30.45
Wpi log 2013.06.08_16.30.45Wpi log 2013.06.08_16.30.45
Wpi log 2013.06.08_16.30.45
aracely-lahermosa
 
Purchased
PurchasedPurchased
Purchased
94240577
 
R000000000005 uj
R000000000005 ujR000000000005 uj
R000000000005 uj
URBANSWAG
 
Sched lgu
Sched lguSched lgu
Sched lgu
nutacthuthu
 
Install
InstallInstall
Install
trungtrau
 
Wpi log 2012.06.26_17.19.20
Wpi log 2012.06.26_17.19.20Wpi log 2012.06.26_17.19.20
Wpi log 2012.06.26_17.19.20
HugoPaco
 
Proxy
ProxyProxy
Forti client00000
Forti client00000Forti client00000
Forti client00000
Alexandre-Toffalini
 
Script
ScriptScript
Script
fjlainfo
 
Cybersecurity200
Cybersecurity200Cybersecurity200
Cybersecurity200
Brent Harrell
 
Otl
OtlOtl
Play claw
Play clawPlay claw
Play claw
Nikolas Aguilera
 

What's hot (17)

Sched lgullll
Sched lgullllSched lgullll
Sched lgullll
 
Viva
VivaViva
Viva
 
Eula
EulaEula
Eula
 
Slow
SlowSlow
Slow
 
Sysinfo
SysinfoSysinfo
Sysinfo
 
Wpi log 2013.06.08_16.30.45
Wpi log 2013.06.08_16.30.45Wpi log 2013.06.08_16.30.45
Wpi log 2013.06.08_16.30.45
 
Purchased
PurchasedPurchased
Purchased
 
R000000000005 uj
R000000000005 ujR000000000005 uj
R000000000005 uj
 
Sched lgu
Sched lguSched lgu
Sched lgu
 
Install
InstallInstall
Install
 
Wpi log 2012.06.26_17.19.20
Wpi log 2012.06.26_17.19.20Wpi log 2012.06.26_17.19.20
Wpi log 2012.06.26_17.19.20
 
Proxy
ProxyProxy
Proxy
 
Forti client00000
Forti client00000Forti client00000
Forti client00000
 
Script
ScriptScript
Script
 
Cybersecurity200
Cybersecurity200Cybersecurity200
Cybersecurity200
 
Otl
OtlOtl
Otl
 
Play claw
Play clawPlay claw
Play claw
 

Similar to On the Value of User Preferences in Search-Based Software Engineering:

Fraglist
FraglistFraglist
Fraglist
CaitlinR
 
DEF CON 24 - Six Volts and Haystack - cheap tools for hacking heavy trucks
DEF CON 24 - Six Volts and Haystack - cheap tools for hacking heavy trucksDEF CON 24 - Six Volts and Haystack - cheap tools for hacking heavy trucks
DEF CON 24 - Six Volts and Haystack - cheap tools for hacking heavy trucks
Felipe Prado
 
Música
MúsicaMúsica
Xirrus Corporate Brochure
Xirrus Corporate BrochureXirrus Corporate Brochure
Xirrus Corporate Brochure
Emanuel Barradas Curto
 
Volumen c final
Volumen c finalVolumen c final
Volumen c final
jhurtado2013
 
Avseq01.dat
Avseq01.datAvseq01.dat
Avseq01.dat
Ngoc Pham
 
Vid 00020 20120316-0350.3 gp
Vid 00020 20120316-0350.3 gpVid 00020 20120316-0350.3 gp
Vid 00020 20120316-0350.3 gp
Gina Paola Paez Gaviria
 
Opun principles training enabling finalpptx
Opun principles training enabling finalpptxOpun principles training enabling finalpptx
Opun principles training enabling finalpptx
OPUNArch
 
Install
InstallInstall
Install
murciadmg
 
Driver whiz serial_key_download
Driver whiz serial_key_downloadDriver whiz serial_key_download
Driver whiz serial_key_download
DANIELFIVE
 
Driver whiz serial_key_download
Driver whiz serial_key_downloadDriver whiz serial_key_download
Driver whiz serial_key_download
DANIELFIVE
 
Tgcmlog
TgcmlogTgcmlog
Tgcmlog
jjpf_1977
 
Informe
InformeInforme
Informe
Jaaz Martinez
 
Wpi log 2013.06.08_16.30.45
Wpi log 2013.06.08_16.30.45Wpi log 2013.06.08_16.30.45
Wpi log 2013.06.08_16.30.45
aracely-lahermosa
 
Closing the Wealth Gap
Closing the Wealth GapClosing the Wealth Gap
Closing the Wealth Gap
ScaleUp Partners LLC
 
4.mpg.media info
4.mpg.media info4.mpg.media info
4.mpg.media info
muhd nor rahimin
 
4.mpg.media info
4.mpg.media info4.mpg.media info
4.mpg.media info
muhd nor rahimin
 
Gannt chart (to do list)
Gannt chart (to do list)Gannt chart (to do list)
Gannt chart (to do list)
Confidential
 
Songs
SongsSongs
Frame tech onsite_seo
Frame tech onsite_seoFrame tech onsite_seo
Frame tech onsite_seo
John Stephenson
 

Similar to On the Value of User Preferences in Search-Based Software Engineering: (20)

Fraglist
FraglistFraglist
Fraglist
 
DEF CON 24 - Six Volts and Haystack - cheap tools for hacking heavy trucks
DEF CON 24 - Six Volts and Haystack - cheap tools for hacking heavy trucksDEF CON 24 - Six Volts and Haystack - cheap tools for hacking heavy trucks
DEF CON 24 - Six Volts and Haystack - cheap tools for hacking heavy trucks
 
Música
MúsicaMúsica
Música
 
Xirrus Corporate Brochure
Xirrus Corporate BrochureXirrus Corporate Brochure
Xirrus Corporate Brochure
 
Volumen c final
Volumen c finalVolumen c final
Volumen c final
 
Avseq01.dat
Avseq01.datAvseq01.dat
Avseq01.dat
 
Vid 00020 20120316-0350.3 gp
Vid 00020 20120316-0350.3 gpVid 00020 20120316-0350.3 gp
Vid 00020 20120316-0350.3 gp
 
Opun principles training enabling finalpptx
Opun principles training enabling finalpptxOpun principles training enabling finalpptx
Opun principles training enabling finalpptx
 
Install
InstallInstall
Install
 
Driver whiz serial_key_download
Driver whiz serial_key_downloadDriver whiz serial_key_download
Driver whiz serial_key_download
 
Driver whiz serial_key_download
Driver whiz serial_key_downloadDriver whiz serial_key_download
Driver whiz serial_key_download
 
Tgcmlog
TgcmlogTgcmlog
Tgcmlog
 
Informe
InformeInforme
Informe
 
Wpi log 2013.06.08_16.30.45
Wpi log 2013.06.08_16.30.45Wpi log 2013.06.08_16.30.45
Wpi log 2013.06.08_16.30.45
 
Closing the Wealth Gap
Closing the Wealth GapClosing the Wealth Gap
Closing the Wealth Gap
 
4.mpg.media info
4.mpg.media info4.mpg.media info
4.mpg.media info
 
4.mpg.media info
4.mpg.media info4.mpg.media info
4.mpg.media info
 
Gannt chart (to do list)
Gannt chart (to do list)Gannt chart (to do list)
Gannt chart (to do list)
 
Songs
SongsSongs
Songs
 
Frame tech onsite_seo
Frame tech onsite_seoFrame tech onsite_seo
Frame tech onsite_seo
 

More from CS, NcState

Talks2015 novdec
Talks2015 novdecTalks2015 novdec
Talks2015 novdec
CS, NcState
 
Future se oct15
Future se oct15Future se oct15
Future se oct15
CS, NcState
 
GALE: Geometric active learning for Search-Based Software Engineering
GALE: Geometric active learning for Search-Based Software EngineeringGALE: Geometric active learning for Search-Based Software Engineering
GALE: Geometric active learning for Search-Based Software Engineering
CS, NcState
 
Big Data: the weakest link
Big Data: the weakest linkBig Data: the weakest link
Big Data: the weakest link
CS, NcState
 
Three Laws of Trusted Data Sharing: (Building a Better Business Case for Dat...
Three Laws of Trusted Data Sharing:(Building a Better Business Case for Dat...Three Laws of Trusted Data Sharing:(Building a Better Business Case for Dat...
Three Laws of Trusted Data Sharing: (Building a Better Business Case for Dat...
CS, NcState
 
Lexisnexis june9
Lexisnexis june9Lexisnexis june9
Lexisnexis june9
CS, NcState
 
Welcome to ICSE NIER’15 (new ideas and emerging results).
Welcome to ICSE NIER’15 (new ideas and emerging results).Welcome to ICSE NIER’15 (new ideas and emerging results).
Welcome to ICSE NIER’15 (new ideas and emerging results).
CS, NcState
 
Icse15 Tech-briefing Data Science
Icse15 Tech-briefing Data ScienceIcse15 Tech-briefing Data Science
Icse15 Tech-briefing Data Science
CS, NcState
 
Kits to Find the Bits that Fits
Kits to Find  the Bits that Fits Kits to Find  the Bits that Fits
Kits to Find the Bits that Fits
CS, NcState
 
Ai4se lab template
Ai4se lab templateAi4se lab template
Ai4se lab template
CS, NcState
 
Automated Software Enging, Fall 2015, NCSU
Automated Software Enging, Fall 2015, NCSUAutomated Software Enging, Fall 2015, NCSU
Automated Software Enging, Fall 2015, NCSU
CS, NcState
 
Requirements Engineering
Requirements EngineeringRequirements Engineering
Requirements Engineering
CS, NcState
 
172529main ken and_tim_software_assurance_research_at_west_virginia
172529main ken and_tim_software_assurance_research_at_west_virginia172529main ken and_tim_software_assurance_research_at_west_virginia
172529main ken and_tim_software_assurance_research_at_west_virginia
CS, NcState
 
Automated Software Engineering
Automated Software EngineeringAutomated Software Engineering
Automated Software Engineering
CS, NcState
 
Next Generation “Treatment Learning” (finding the diamonds in the dust)
Next Generation “Treatment Learning” (finding the diamonds in the dust)Next Generation “Treatment Learning” (finding the diamonds in the dust)
Next Generation “Treatment Learning” (finding the diamonds in the dust)
CS, NcState
 
Tim Menzies, directions in Data Science
Tim Menzies, directions in Data ScienceTim Menzies, directions in Data Science
Tim Menzies, directions in Data Science
CS, NcState
 
Goldrush
GoldrushGoldrush
Goldrush
CS, NcState
 
Dagstuhl14 intro-v1
Dagstuhl14 intro-v1Dagstuhl14 intro-v1
Dagstuhl14 intro-v1
CS, NcState
 
Know thy tools
Know thy toolsKnow thy tools
Know thy tools
CS, NcState
 
The Art and Science of Analyzing Software Data
The Art and Science of Analyzing Software DataThe Art and Science of Analyzing Software Data
The Art and Science of Analyzing Software Data
CS, NcState
 

More from CS, NcState (20)

Talks2015 novdec
Talks2015 novdecTalks2015 novdec
Talks2015 novdec
 
Future se oct15
Future se oct15Future se oct15
Future se oct15
 
GALE: Geometric active learning for Search-Based Software Engineering
GALE: Geometric active learning for Search-Based Software EngineeringGALE: Geometric active learning for Search-Based Software Engineering
GALE: Geometric active learning for Search-Based Software Engineering
 
Big Data: the weakest link
Big Data: the weakest linkBig Data: the weakest link
Big Data: the weakest link
 
Three Laws of Trusted Data Sharing: (Building a Better Business Case for Dat...
Three Laws of Trusted Data Sharing:(Building a Better Business Case for Dat...Three Laws of Trusted Data Sharing:(Building a Better Business Case for Dat...
Three Laws of Trusted Data Sharing: (Building a Better Business Case for Dat...
 
Lexisnexis june9
Lexisnexis june9Lexisnexis june9
Lexisnexis june9
 
Welcome to ICSE NIER’15 (new ideas and emerging results).
Welcome to ICSE NIER’15 (new ideas and emerging results).Welcome to ICSE NIER’15 (new ideas and emerging results).
Welcome to ICSE NIER’15 (new ideas and emerging results).
 
Icse15 Tech-briefing Data Science
Icse15 Tech-briefing Data ScienceIcse15 Tech-briefing Data Science
Icse15 Tech-briefing Data Science
 
Kits to Find the Bits that Fits
Kits to Find  the Bits that Fits Kits to Find  the Bits that Fits
Kits to Find the Bits that Fits
 
Ai4se lab template
Ai4se lab templateAi4se lab template
Ai4se lab template
 
Automated Software Enging, Fall 2015, NCSU
Automated Software Enging, Fall 2015, NCSUAutomated Software Enging, Fall 2015, NCSU
Automated Software Enging, Fall 2015, NCSU
 
Requirements Engineering
Requirements EngineeringRequirements Engineering
Requirements Engineering
 
172529main ken and_tim_software_assurance_research_at_west_virginia
172529main ken and_tim_software_assurance_research_at_west_virginia172529main ken and_tim_software_assurance_research_at_west_virginia
172529main ken and_tim_software_assurance_research_at_west_virginia
 
Automated Software Engineering
Automated Software EngineeringAutomated Software Engineering
Automated Software Engineering
 
Next Generation “Treatment Learning” (finding the diamonds in the dust)
Next Generation “Treatment Learning” (finding the diamonds in the dust)Next Generation “Treatment Learning” (finding the diamonds in the dust)
Next Generation “Treatment Learning” (finding the diamonds in the dust)
 
Tim Menzies, directions in Data Science
Tim Menzies, directions in Data ScienceTim Menzies, directions in Data Science
Tim Menzies, directions in Data Science
 
Goldrush
GoldrushGoldrush
Goldrush
 
Dagstuhl14 intro-v1
Dagstuhl14 intro-v1Dagstuhl14 intro-v1
Dagstuhl14 intro-v1
 
Know thy tools
Know thy toolsKnow thy tools
Know thy tools
 
The Art and Science of Analyzing Software Data
The Art and Science of Analyzing Software DataThe Art and Science of Analyzing Software Data
The Art and Science of Analyzing Software Data
 

Recently uploaded

Introduction to Banking System in India.ppt
Introduction to Banking System in India.pptIntroduction to Banking System in India.ppt
Introduction to Banking System in India.ppt
Dr. S. Bulomine Regi
 
BÀI TẬP BỔ TRỢ 4 KỸ NĂNG TIẾNG ANH LỚP 12 - GLOBAL SUCCESS - FORM MỚI 2025 - ...
BÀI TẬP BỔ TRỢ 4 KỸ NĂNG TIẾNG ANH LỚP 12 - GLOBAL SUCCESS - FORM MỚI 2025 - ...BÀI TẬP BỔ TRỢ 4 KỸ NĂNG TIẾNG ANH LỚP 12 - GLOBAL SUCCESS - FORM MỚI 2025 - ...
BÀI TẬP BỔ TRỢ 4 KỸ NĂNG TIẾNG ANH LỚP 12 - GLOBAL SUCCESS - FORM MỚI 2025 - ...
Nguyen Thanh Tu Collection
 
FINAL MATATAG Kindergarten CG 2023 pdf
FINAL MATATAG Kindergarten CG 2023   pdfFINAL MATATAG Kindergarten CG 2023   pdf
FINAL MATATAG Kindergarten CG 2023 pdf
maricelrudela
 
JavaScript Interview Questions PDF By ScholarHat
JavaScript Interview  Questions PDF By ScholarHatJavaScript Interview  Questions PDF By ScholarHat
JavaScript Interview Questions PDF By ScholarHat
Scholarhat
 
Open Source and AI - ByWater Closing Keynote Presentation.pdf
Open Source and AI - ByWater Closing Keynote Presentation.pdfOpen Source and AI - ByWater Closing Keynote Presentation.pdf
Open Source and AI - ByWater Closing Keynote Presentation.pdf
Jessica Zairo
 
View Inheritance in Odoo 17 - Odoo 17 Slides
View Inheritance in Odoo 17 - Odoo 17  SlidesView Inheritance in Odoo 17 - Odoo 17  Slides
View Inheritance in Odoo 17 - Odoo 17 Slides
Celine George
 
ASP.NET Core Interview Questions PDF By ScholarHat.pdf
ASP.NET Core Interview Questions PDF By ScholarHat.pdfASP.NET Core Interview Questions PDF By ScholarHat.pdf
ASP.NET Core Interview Questions PDF By ScholarHat.pdf
Scholarhat
 
Genetics Teaching Plan: Dr.Kshirsagar R.V.
Genetics Teaching Plan: Dr.Kshirsagar R.V.Genetics Teaching Plan: Dr.Kshirsagar R.V.
Genetics Teaching Plan: Dr.Kshirsagar R.V.
DrRavindrakshirsagar1
 
matatag curriculum education for Kindergarten
matatag curriculum education for Kindergartenmatatag curriculum education for Kindergarten
matatag curriculum education for Kindergarten
SarahAlie1
 
Mail Server Configuration Using App passwords in Odoo 17
Mail Server Configuration Using App passwords in Odoo 17Mail Server Configuration Using App passwords in Odoo 17
Mail Server Configuration Using App passwords in Odoo 17
Celine George
 
2 Post harvest Physiology of Horticulture produce.pptx
2 Post harvest Physiology of Horticulture  produce.pptx2 Post harvest Physiology of Horticulture  produce.pptx
2 Post harvest Physiology of Horticulture produce.pptx
UmeshTimilsina1
 
How to Manage Line Discount in Odoo 17 POS
How to Manage Line Discount in Odoo 17 POSHow to Manage Line Discount in Odoo 17 POS
How to Manage Line Discount in Odoo 17 POS
Celine George
 
What is Rescue Session in Odoo 17 POS - Odoo 17 Slides
What is Rescue Session in Odoo 17 POS - Odoo 17 SlidesWhat is Rescue Session in Odoo 17 POS - Odoo 17 Slides
What is Rescue Session in Odoo 17 POS - Odoo 17 Slides
Celine George
 
formative Evaluation By Dr.Kshirsagar R.V
formative Evaluation By Dr.Kshirsagar R.Vformative Evaluation By Dr.Kshirsagar R.V
formative Evaluation By Dr.Kshirsagar R.V
DrRavindrakshirsagar1
 
NAEYC Code of Ethical Conduct Resource Book
NAEYC Code of Ethical Conduct Resource BookNAEYC Code of Ethical Conduct Resource Book
NAEYC Code of Ethical Conduct Resource Book
lakitawilson
 
Power of Ignored Skills: Change the Way You Think and Decide by Manoj Tripathi
Power of Ignored Skills: Change the Way You Think and Decide by Manoj TripathiPower of Ignored Skills: Change the Way You Think and Decide by Manoj Tripathi
Power of Ignored Skills: Change the Way You Think and Decide by Manoj Tripathi
Pankaj523992
 
How To Update One2many Field From OnChange of Field in Odoo 17
How To Update One2many Field From OnChange of Field in Odoo 17How To Update One2many Field From OnChange of Field in Odoo 17
How To Update One2many Field From OnChange of Field in Odoo 17
Celine George
 
C# Interview Questions PDF By ScholarHat.pdf
C# Interview Questions PDF By ScholarHat.pdfC# Interview Questions PDF By ScholarHat.pdf
C# Interview Questions PDF By ScholarHat.pdf
Scholarhat
 
Parent PD Design for Professional Development .docx
Parent PD Design for Professional Development .docxParent PD Design for Professional Development .docx
Parent PD Design for Professional Development .docx
AntonioJarligoCompra
 
SEQUNCES Lecture_Notes_Unit4_chapter11_sequence
SEQUNCES  Lecture_Notes_Unit4_chapter11_sequenceSEQUNCES  Lecture_Notes_Unit4_chapter11_sequence
SEQUNCES Lecture_Notes_Unit4_chapter11_sequence
Murugan Solaiyappan
 

Recently uploaded (20)

Introduction to Banking System in India.ppt
Introduction to Banking System in India.pptIntroduction to Banking System in India.ppt
Introduction to Banking System in India.ppt
 
BÀI TẬP BỔ TRỢ 4 KỸ NĂNG TIẾNG ANH LỚP 12 - GLOBAL SUCCESS - FORM MỚI 2025 - ...
BÀI TẬP BỔ TRỢ 4 KỸ NĂNG TIẾNG ANH LỚP 12 - GLOBAL SUCCESS - FORM MỚI 2025 - ...BÀI TẬP BỔ TRỢ 4 KỸ NĂNG TIẾNG ANH LỚP 12 - GLOBAL SUCCESS - FORM MỚI 2025 - ...
BÀI TẬP BỔ TRỢ 4 KỸ NĂNG TIẾNG ANH LỚP 12 - GLOBAL SUCCESS - FORM MỚI 2025 - ...
 
FINAL MATATAG Kindergarten CG 2023 pdf
FINAL MATATAG Kindergarten CG 2023   pdfFINAL MATATAG Kindergarten CG 2023   pdf
FINAL MATATAG Kindergarten CG 2023 pdf
 
JavaScript Interview Questions PDF By ScholarHat
JavaScript Interview  Questions PDF By ScholarHatJavaScript Interview  Questions PDF By ScholarHat
JavaScript Interview Questions PDF By ScholarHat
 
Open Source and AI - ByWater Closing Keynote Presentation.pdf
Open Source and AI - ByWater Closing Keynote Presentation.pdfOpen Source and AI - ByWater Closing Keynote Presentation.pdf
Open Source and AI - ByWater Closing Keynote Presentation.pdf
 
View Inheritance in Odoo 17 - Odoo 17 Slides
View Inheritance in Odoo 17 - Odoo 17  SlidesView Inheritance in Odoo 17 - Odoo 17  Slides
View Inheritance in Odoo 17 - Odoo 17 Slides
 
ASP.NET Core Interview Questions PDF By ScholarHat.pdf
ASP.NET Core Interview Questions PDF By ScholarHat.pdfASP.NET Core Interview Questions PDF By ScholarHat.pdf
ASP.NET Core Interview Questions PDF By ScholarHat.pdf
 
Genetics Teaching Plan: Dr.Kshirsagar R.V.
Genetics Teaching Plan: Dr.Kshirsagar R.V.Genetics Teaching Plan: Dr.Kshirsagar R.V.
Genetics Teaching Plan: Dr.Kshirsagar R.V.
 
matatag curriculum education for Kindergarten
matatag curriculum education for Kindergartenmatatag curriculum education for Kindergarten
matatag curriculum education for Kindergarten
 
Mail Server Configuration Using App passwords in Odoo 17
Mail Server Configuration Using App passwords in Odoo 17Mail Server Configuration Using App passwords in Odoo 17
Mail Server Configuration Using App passwords in Odoo 17
 
2 Post harvest Physiology of Horticulture produce.pptx
2 Post harvest Physiology of Horticulture  produce.pptx2 Post harvest Physiology of Horticulture  produce.pptx
2 Post harvest Physiology of Horticulture produce.pptx
 
How to Manage Line Discount in Odoo 17 POS
How to Manage Line Discount in Odoo 17 POSHow to Manage Line Discount in Odoo 17 POS
How to Manage Line Discount in Odoo 17 POS
 
What is Rescue Session in Odoo 17 POS - Odoo 17 Slides
What is Rescue Session in Odoo 17 POS - Odoo 17 SlidesWhat is Rescue Session in Odoo 17 POS - Odoo 17 Slides
What is Rescue Session in Odoo 17 POS - Odoo 17 Slides
 
formative Evaluation By Dr.Kshirsagar R.V
formative Evaluation By Dr.Kshirsagar R.Vformative Evaluation By Dr.Kshirsagar R.V
formative Evaluation By Dr.Kshirsagar R.V
 
NAEYC Code of Ethical Conduct Resource Book
NAEYC Code of Ethical Conduct Resource BookNAEYC Code of Ethical Conduct Resource Book
NAEYC Code of Ethical Conduct Resource Book
 
Power of Ignored Skills: Change the Way You Think and Decide by Manoj Tripathi
Power of Ignored Skills: Change the Way You Think and Decide by Manoj TripathiPower of Ignored Skills: Change the Way You Think and Decide by Manoj Tripathi
Power of Ignored Skills: Change the Way You Think and Decide by Manoj Tripathi
 
How To Update One2many Field From OnChange of Field in Odoo 17
How To Update One2many Field From OnChange of Field in Odoo 17How To Update One2many Field From OnChange of Field in Odoo 17
How To Update One2many Field From OnChange of Field in Odoo 17
 
C# Interview Questions PDF By ScholarHat.pdf
C# Interview Questions PDF By ScholarHat.pdfC# Interview Questions PDF By ScholarHat.pdf
C# Interview Questions PDF By ScholarHat.pdf
 
Parent PD Design for Professional Development .docx
Parent PD Design for Professional Development .docxParent PD Design for Professional Development .docx
Parent PD Design for Professional Development .docx
 
SEQUNCES Lecture_Notes_Unit4_chapter11_sequence
SEQUNCES  Lecture_Notes_Unit4_chapter11_sequenceSEQUNCES  Lecture_Notes_Unit4_chapter11_sequence
SEQUNCES Lecture_Notes_Unit4_chapter11_sequence
 

On the Value of User Preferences in Search-Based Software Engineering: