Agent-Based Configuration of (Metaheuristic) Algorithms - Doctoral dissertation

1. Agent-Based Configuration of (Metaheuristic) Algorithms - Doctoral dissertation - M.Sc. Dagmar Monett Díaz Artificial Intelligence, Computer Science Dept. Humboldt University of Berlin 25.02.2005

2. Outline • Introduction – Metaheuristics; Configuration process; Why agents? – PhD: Motivation – Goals - Contributions • Agent-Based Configuration of (Metaheuristic) Algorithms – Architecture; agents; interaction protocols; other details • Experimental results • Ongoing projects • Conclusions and Contributions D. Monett Díaz Humboldt University of Berlin Doctoral dissertation, 25.02.2005

4. Metaheuristics A metaheuristic is “a master strategy that guides and modifies other heuristics (like local search procedures) to produce solutions beyond those that are normally generated in a quest for local optimality“ [Laguna 2002] Examples of metaheuristics: - Traditional approaches: •EC (Evolutionary Computation): GA (Genetic Algorithms), ES (Evolution Strategies), GP (Genetic Programming), etc. •SA (Simulated Annealing), TS (Tabu Search), ANN (Artificial Neural Networks), EDA (Estimation of Distribution Algorithms), ACS (Ant Colony Systems), etc. - Hybrid metaheuristics  recent approaches! New research goals in metaheuristics domain : - To combine aspects from different metaheuristics, Artificial Intelligence, Operations Research techniques, etc. - Experimental design for configuring is important - Optimization of parameters (i.e. configuration process) is a relevant issue D. Monett Díaz Humboldt University of Berlin Doctoral dissertation, 25.02.2005

5. Configuration of algorithms (or “fine-tuning” of algorithms) Shortcomings: Not all metaheuristic algorithms are auto-adaptive (in particular the hybrid approaches) Usually, control parameters are set by hand or in the spirit of brute-force mechanisms; time-consuming task Very few published research works; not yet an established research area Distributed, remote or parallel execution of configuration algorithms: not existing Special topic in most recent conferences and workshops (e.g. HM’04 at ECAI’04 and PSGEA’05 at GECCO’05); current open question!! D. Monett Díaz Humboldt University of Berlin Doctoral dissertation, 25.02.2005

6. Example: a Tabu Search Algorithm • Fixed parameters: - Tabu criterion (what is forbidden for a given number of iterations) - Nr. of elite solutions (intensification strategy: restarting the search around them) - Stop criterion (e.g. maximum number of iterations) • Free parameter (i.e. factor in study): Tabu list length or tabu tenure - Levels: 10, 20, 50  3 configurations • Number of global runs (without varying any parameter): NTrials one TS 1 TTenure=10 1 TTenure=20 1 TTenure=50 run!! & & & fixed factors fixed factors fixed factors . . . . + . + . . . . NTrials TTenure=10 NTrials TTenure=20 NTrials TTenure=50 & & & fixed factors fixed factors fixed factors ► Which is the “most acceptable” configuration? D. Monett Díaz Humboldt University of Berlin Doctoral dissertation, 25.02.2005

7. Why agents for configuring? Could agents substitute humans in their tasks? Could they behave as experts would do? E.g. • by alleviating the time required while configuring, i.e., the cost of time-intensive fine-tuning? • by entering data and processing results? • by autonomously conducting the required experiments? • by (semi) automating the configuration process? Answer: Agent-based configuration !! Multiagent Systems: properties that are of interest - Distributed / Remote execution - Cooperation among the agents - Autonomy & Specialization - Collaborative exploration / exploitation of the search space D. Monett Díaz Humboldt University of Berlin Doctoral dissertation, 25.02.2005

8. Motivation – Goals – Contributions user problems Motivation:  Powerful algorithms are needed to (metaheuristic) algorithms solve several real problems  Configuring them can be a very difficult combinatorial problem D. Monett Díaz Humboldt University of Berlin Doctoral dissertation, 25.02.2005

9. Motivation – Goals – Contributions user problems Goals:  To support the configuration (metaheuristic) algorithms process of these algorithms . Autonomously, distributed, remote, etc.  To (semi)automate both monitoring and fine-tuning of parameters and conducting experiments D. Monett Díaz Humboldt University of Berlin Doctoral dissertation, 25.02.2005

10. Motivation – Goals – Contributions user problems +CARPS Contributions:  +CARPS, agent-based approach (metaheuristic) algorithms for configuring . Distributed, collaborative problem solving . Necessary information specialization/processing . Flexibility: very important D. Monett Díaz Humboldt University of Berlin Doctoral dissertation, 25.02.2005

12. +CARPS : Architecture +CARPS : Multi-Agent System for Configuring Algorithms in Real Problem Solving +CARPS I/O Layer Algorithm Configuration Layer Algorithm Solution Layer Communication Layer D. Monett (2004). Collaborative JADE Agents Enabling the Configuration of Algorithms. In Proceedings of the International Conference on Advances in Intelligent Systems – Theory and Applications, AISTA'2004, IEEE Computer Society, University of Canberra and CRP Henri Tudor, Luxembourg-Kirchberg, Luxembourg. D. Monett (2004). +CARPS: Configuration of Metaheuristics Based on Cooperative Agents. In Proceedings of the First International Workshop on Hybrid Metaheuristics, HM'2004, at the 16th European Conference on Artificial Intelligence, ECAI'2004, Valencia, Spain. D. Monett Díaz Humboldt University of Berlin Doctoral dissertation, 25.02.2005

13. +CARPS : Agent interactions Pf , Px , ag. lists GUI MAS-User UM interaction Pf , ag. lists best ag. lists solutions paths to M ABRRHC and i/o files info and SM par. names, ag. lists ag. lists exch. solutions solutions data configs configs M solutions AC AS solutions TS, GA, ES, etc. STi initial results configs Algorithm’s execution: optimization process STi (user problem’s parameters) ISM SCB Configuration process: optimization process (metaheuristic’s parameters) D. Monett Díaz Humboldt University of Berlin Doctoral dissertation, 25.02.2005

14. Agent-Based Configuration Algorithm Configuration algorithm: Agent-Based Random-Restart Hill-Climbing Particularities: • Stochastic Hill-Climbing algorithm (restart from different candidate solutions) • Well-known metaheuristic; easy to implement • Construction of a topology (neighbors per solution) • Migration policy (AC agents exchange best-so-far obtained solutions) • ABRRHC: itself a hybrid metaheuristic Implementing a different only few changes in configuration algorithm specialized agent!!! D. Monett Díaz Humboldt University of Berlin Doctoral dissertation, 25.02.2005

15. Worth of a metaheuristic Factors used to analyze metaheuristic’s performance: • quality of the best solution found until a stop criterion is verified • time to get the best solution • algorithm's time to reach an “acceptable“ solution • number of function evaluations performed until a stop criterion is verified, etc. Best-so-far configuration • a worth equation indicates “how good” the metaheuristic is • more than one factor at a time is considered • minimization procedure Example of worth equation (using a weighted sum approach) worth( p )  w1  f s  w2  f t  w3  f v Consider quality of the solutions, time to get fx them and evaluations & Normalization, e.g.: for all repetitions with the max( f s , f t , f v ) same configuration (i.e. p) D. Monett Díaz Humboldt University of Berlin Doctoral dissertation, 25.02.2005

16. Agents’ communications Interaction Protocols (IPs) • Define typical patterns of message exchange between agents • Agents are supposed to behave consistently by following such IPs • FIPA standards (e.g. Contract Net IP) • New applications might need new IPs – E.g. agent-based configuration of (metaheuristic) algorithms • In +CARPS: IPs are implemented as FSM Behaviors from JADE • Agent communications: ACL messages, FIPA compliant D. Monett (2004). Interaction Protocols for +CARPS Agents: Booking and Getting Engaged for Configuring. In Proceedings of the Workshop Concurrency, Specification, and Programming CS&P'2004, volume 3: Multiagent Systems and Applications, Caputh, Germany (Also in Special Issue of Fundamenta Informaticae –to appear–) D. Monett Díaz Humboldt University of Berlin Doctoral dissertation, 25.02.2005

17. Agents’ communications Most relevant IPs in +CARPS Helper-Booker-Protocol AS, AC UM Initiator Participant Interaction Protocols propose reject-proposal Helper-Booker IP failure accept-proposal Engagement IP inform failure [dead- Request IP line1] confirm cancel [deadline2] - Which agents will participate? - Among UM agent (booker) and AC and AS agents (helpers) - UM registers with the Directory Facilitator - Helpers are initiators of the HBIPs; bookers are responders D. Monett Díaz Humboldt University of Berlin Doctoral dissertation, 25.02.2005

18. +CARPS’ Graphical User Interface

24. Metaheuristics that are considered Genetic Algorithm (GA) • GA optimal parameters  unknown • Parameter estimation in chemical reactions (copolymerizations) D. Monett, J.A. Méndez, G.A. Abraham, A. Gallardo, J. San Román (2002). An Evolutionary Approach to Reactivity Ratios Prediction. Macromol. Theory Simul., 11(5):525-532. Wiley-VCH Verlag GmbH, Weinheim, Germany. D. Monett (2001). On the automation of evolutionary techniques and their application to inverse problems from Chemical Kinetics. In Proceedings of the GECCO'01 Graduate Student Workshop, Genetic and Evolutionary Computation Conference, San Francisco, California, USA. • Data provided by colleagues from Institute of Polymer Science and Technology, Superior Council of Scientific Research, Spain, and Biomaterials Center, Havana University, Cuba Evolution Strategy (ES) • ES optimal parameters  known • Function optimization (Rechenberg 94) • Data and program provided by I. Santibáñez-Koref, FG Bionik und Evolutionstechnik , TU Berlin D. Monett Díaz Humboldt University of Berlin Doctoral dissertation, 25.02.2005

25. Example: Configuring an ES Evolution Strategy (1, )-ES One parent Number of offspring on each generation (the only individuals that undergo selection) • The ES itself optimizes a quadratic function (sphere model) • ES efficiency: progress to the optimum on each generation • Fixed parameters:  = 10000,  = 1e-30, t = 240 • Free parameters:  and c • Theoretical optima: (, c) = (5, 1) and (5, -1) Experimental settings 60 search trials (ABRRHC) 4 neighbors / configuration 4 solutions to exchange among the AC agents proportion of AC agents / parameter to fine-tune = 2 D. Monett Díaz Humboldt University of Berlin Doctoral dissertation, 25.02.2005

26. Obtained solutions Solution qualities for varying control parameters Total of best singular solutions (including all trials or repetitions) = 2 104 Example of an “acceptable” singular solution (according to its quality) (, c) = (5.15965245, 0.87023895) 0,004 5 4 0,003 3 quality quality 0,002 2 0,001 1 0 0 1,00 3,00 5,00 7,00 9,00 -9,97 -7,15 -4,77 -2,42 -0,73 1,70 3,90 6,63 9,21 lambda c D. Monett Díaz Humboldt University of Berlin Doctoral dissertation, 25.02.2005

27. Pareto-optimal singular solutions 60 50 40 Weight vector quality 30 (w1, w2, w3) = (1.0, 0.0, 0.0) 20 10 0 The importance goes to the solution qualities 0 1000 2000 3000 4000 5000 6000 time (msec) (when calculating the worth) 60 50 Calculated Pareto-optimal solutions, 40 according to all criteria, quality 30 are presented to the user 20 (GUI + data files) 10 0 0 5000 10000 15000 20000 25000 30000 35000 func.eval. D. Monett Díaz Humboldt University of Berlin Doctoral dissertation, 25.02.2005

28. Configuration process Estimated time per agent: AC 32,8282333 min 800 AS 3,25705868 hours 600 % from the total: Configurators 14,382467 400 Solvers 85,617533 200 Total of algorithm’s evaluations = 10 520 0 ABRRHC Solver (i.e. ES evaluations) time (min) 131,3129333 781,6940833 as all AC agents would as all AS agents would sequentially run sequentially run Other experiments already done with ES and GA: - varying the worth equation and the weight vectors - varying the proportion of agents, exchanges, neighbors per solution, etc. - studying IPs (e.g. HBIP, EIP, RequestIP) in detail D. Monett Díaz Humboldt University of Berlin Doctoral dissertation, 25.02.2005

30. Ongoing projects • Optimizing risk strategies in random investment environments (GA, 4 free parameters, 2x agents) (E. Navarro, D. Monett, V. Uc Cetina: Machine Learning Approaches for Investment Strategies --- working paper) • Adapting soccer agents’ movement, RoboCup 3D Simulation League (ANN, 50 free parameters, 1x & 2x agents) New to +CARPS: - Connection Clients or CC agents - TCP/IP communication instead data files transfer - related ontologies, classes, and agent behaviors and interactions • Including other configuration algorithms D. Monett Díaz Humboldt University of Berlin Doctoral dissertation, 25.02.2005

32. Conclusions and Contributions +CARPS … is an agent-based approach to the configuration of algorithms including, but not limited to, metaheuristics is a framework that helps monitoring control factors of metaheuristics is a tool useful for conducting experiments when executing these algorithms follows a theoretical description and formalization of the configuration problem D. Monett (2003). Configuration of Metaheuristics: Overview and Theoretical Approach. In Proceedings of the Workshop Concurrency, Specification, and Programming CS&P'2003, volume 2, Czarna, Poland. D. Monett Díaz Humboldt University of Berlin Doctoral dissertation, 25.02.2005

33. Conclusions and Contributions +CARPS … consists of different types of autonomous, cooperative agents that support the configuration of metaheuristic algorithms in a distributed fashion implements a Random-Restart Hill-Climbing algorithm to search for solutions during the configuration process includes new interaction protocols that are followed by the agents in order to cover communication requirements needed in the domain of analysis is a prototype implemented over JADE that follows FIPA specifications provides an infrastructure for a distributed, remote and parallel execution of configuration algorithms (i.e. ontologies, interaction protocols, agent behaviors, and other supporting classes) D. Monett Díaz Humboldt University of Berlin Doctoral dissertation, 25.02.2005

34. Agent-Based Configuration of (Metaheuristic) Algorithms - Doctoral dissertation - M.Sc. Dagmar Monett Díaz Artificial Intelligence, Computer Science Dept. Humboldt University of Berlin 25.02.2005

35. Extra, additional slides - Used in answers, examples, discussion, etc. - M.Sc. Dagmar Monett Díaz Artificial Intelligence, Computer Science Dept. Humboldt University of Berlin 25.02.2005

36. “Fine-tuning” of metaheuristics Auto-adaptive metaheuristics… Brute force - Every configuration is tested and the best one is selected Meta-evolution as “configurator” - Evolutionary algorithms fine-tune other evolutionary algorithms - Examples: meta-level Evo.Alg. GA that produces ES+GA meta-solutions They optimize It optimizes a benchmark single case of functions GA GA the sphere model [Grefenstette, 1986] [Bäck, 1994] [Pham, 1995] [Pedroso, 1997] D. Monett Díaz Humboldt University of Berlin Doctoral dissertation, 25.02.2005

37. More general “configurators” Using experimental design - Statistical design + gradient descent [Coy et al., 2000] (two local search heuristics) - Fractional experimental design + local search [Adenso-Díaz and Laguna, 2003] (Tabu Search and Simulated Annealing) Racing algorithms - Sequentially evaluate candidate configurations and discard poor ones as soon as statistically sufficient evidence is gathered against them - Example: F-Race: based on a statistical method for hypothesis testing [Birattari et al., 2002] (MAX-MIN-Ant System) Parallel approaches - Independent runs: master-slave approach [Blesa and Xhafa, 2004] (Tabu Search) - Parallel implementations of metaheuristics have also contributed to the topic… D. Monett Díaz Humboldt University of Berlin Doctoral dissertation, 25.02.2005

38. Auto-adaptive metaheuristics Examples of mechanisms that are used: • Genetic operators encoded as part of the representation of individuals (e.g. ES, first Evo.Alg. that considered self-adaptation) • Variable population size / chromosomal length (e.g. GA) • Mutation schedules for varying the mutation of individuals at early and later evolution stages differently (e.g. GA) E.g.: non-uniform mutation r : random in [0, 1] v  (t ,UBk  vk ), r  0.5 T : max. generation number v 'k   k b : system parameter determining  vk   (t , vk  LBk ), r  0.5 the degree of dependency on (1 t )b iteration number t (e.g. b=5)  (t , y )  y  (1  r T ) LB, UB: upper & lower bounds • Cooling schedules for controlling the search and the probability of accepting worsening solutions (e.g. SA) t : temperature E.g.: geometric and arithmetic cooling r : random in [0, 1] f ( s ) f ( s ') t  t   ,  0 s, s’: solutions e t r ? t  t   ,0    1 f: solution quality D. Monett Díaz Humboldt University of Berlin Doctoral dissertation, 25.02.2005

39. Constructing neighbor configurations AC agents construct neighbor configurations ISM agent manages the instantiation strategies Example: The new parameter value is randomly generated in a neighborhood of the current value, with NeighborsPercent sizeN  index ( currentNei ghbor ) sizeN : size of the neighborhood NeighborsPercent : percent of the original restriction to the free parameter index(currentNeighbor) : number of neighbors already constructed + 1 • As the number of neighbors increases, it decreases the ratio of the neighborhood around the current parameter value • Decreasing the neighborhood size adds lower noise to the current parameter value • Considering other adaptive techniques  local to the ISM agent !! D. Monett Díaz Humboldt University of Berlin Doctoral dissertation, 25.02.2005

40. Stop criteria TS 1 TTenure=10 1 TTenure=20 1 TTenure=50 & & & fixed factors fixed factors fixed factors . . . NTrials = ?? . . + . . + . . NTrials TTenure=10 NTrials TTenure=20 NTrials TTenure=50 & & & fixed factors fixed factors fixed factors • Metaheuristics are not exact methods • Goal: To find “acceptable” solutions beyond local optima • Challenge: In less trials as possible • One approach: Stop when the fluctuations of averaged solution qualities with the same configuration is not significant any more (-solution “stability”): i i 1 s s j 1 j j 1 j si  si 1     NTrials = i+1 i i 1 D. Monett Díaz Humboldt University of Berlin Doctoral dissertation, 25.02.2005

41. Stability transition trial Other performance indexes that could be considered: - CPU time to get the solutions - number of function evaluations Related definitions: -computational time “stability” -evaluations “stability” However, it can happen that NTrials for -solution “stability”  NTrials for -computational time “stability”  NTrials for -evaluations “stability” Stability transition trial Τ: Minimal number of trials at which the metaheuristic is solution, computational time and evaluations stable when evaluated with the same configuration D. Monett Díaz Humboldt University of Berlin Doctoral dissertation, 25.02.2005

42. Expectation value From Experimental Physics: • Given: N independent measurements of a physical constant • Mean of the measurements (“estimate” of the true value): N 1 y N yi 1 i N 1 • Expectation value or true value:   y  lim N  N yi 1 i Expectation value of the solution qualities from the program outputs y(∙) when evaluating the metaheuristic with the same configuration, say p T 1 Correction: E y ( p) s   y( p) s  lim s i T: stability transition trial T  T i 1 However, y(∙) may be not (and similarly for computational time, function normally distributed… evaluations or any other performance index…) D. Monett Díaz Humboldt University of Berlin Doctoral dissertation, 25.02.2005

43. Expectation value Idea: what to expect from repeated outcomes • Discrete random variable X with values x1, x2, …, xN • Probability function f(xi)=P(X= xi) (probability of obtaining each xi) • Expectation value: Special case: If f ( x )  1 N i N E X    xi  f ( xi ) then N 1 i 1 E X   x i X N i 1 Expectation value of the solution qualities from the program outputs when evaluating the metaheuristic with the same configuration (and similarly for computational time, function evaluations or any other performance index…) • Probability function is unknown • It can be estimated after the trials are done, i.e. by simulation • Like this: Relative frequencies could be tabulated. They are estimates of the probabilities 1 Correction: f ( si )  is difficult to have… T D. Monett Díaz Humboldt University of Berlin Doctoral dissertation, 25.02.2005

44. Goal of the experiments: Twofold Testing the system  How to use +CARPS: conducting experiments & monitoring the configuration process • Aim: to seek for optimum values to the user problem’s parameters • User problem: Chemical optimization problem (Parameter estimation in chemical reactions) • Former obtained solutions are reproduced; better ones are found Evaluating the system  How does +CARPS work: functionality of the system • Aim: to seek for optimum values to the algorithm’s parameters • Configuration algorithm is tested • Also communication among the agents, interaction protocols, etc. • User problem: of secondary importance at this level D. Monett Díaz Humboldt University of Berlin Doctoral dissertation, 25.02.2005

45. Technical information +CARPS (by August 2004) Agent framework: JADE 3.2 Programming language: JavaTM 2 SDK Standard Ed. 1.4.2 Development tool: BlueJ 2.0 beta Source code lines:  22 500 (most important) Classes 113 (.java) GAs (by 2002) Programming language: Visual C++ 5.0 Development tool: Microsoft Developer Studio 97 Source code lines:  6 000 Total of classes: 10 (.cpp) + 10 headers (.hpp) D. Monett Díaz Humboldt University of Berlin Doctoral dissertation, 25.02.2005

46. Agent-based configuration Optimization Simulation Optimization Agent-based Algorithm Algorithm configuration execution functioning D. Monett Díaz Humboldt University of Berlin Doctoral dissertation, 25.02.2005

47. +CARPS : Development • software development: it was carried out in an evolutionary, object-oriented fashion • bottom-up strategy: behaviors and agent actions and objects were designed, implemented, debugged, and tested as the needs arose, from simple to more complex components. For example, a draft for the configuration ontology was first considered which later turned into the four vocabularies and ontologies • GUI: it was very important for testing agent interactions and their functioning • JADE (Java Agent DEvelopment Framework) – FIPA specifications • +CARPS classes are Java classes D. Monett Díaz Humboldt University of Berlin Doctoral dissertation, 25.02.2005

48. Motivation – Goals – Contributions Real problem user problems Motivation: (metaheuristic) algorithms  Powerful algorithms are needed to solve several real problems Developer  Configuring them can be a very difficult combinatorial problem System D. Monett Díaz Humboldt University of Berlin Doctoral dissertation, 25.02.2005

49. Motivation – Goals – Contributions Real problem user problems Goals: (metaheuristic) algorithms  To support the configuration process of these algorithms Developer . Autonomously, distributed, remote, etc.  To (semi)automate both monitoring and System fine-tuning of parameters and conducting experiments D. Monett Díaz Humboldt University of Berlin Doctoral dissertation, 25.02.2005

50. Motivation – Goals – Contributions Real problem +CARPS user problems Contributions: (metaheuristic) algorithms  +CARPS, agent-based approach for configuring . Distributed, collaborative problem solving Developer . Necessary information specialization/processing System . Flexibility: important D. Monett Díaz Humboldt University of Berlin Doctoral dissertation, 25.02.2005

51. Agent-based Configuration: Steps I: Initializationof variables to manage solutions, agent Initialization of variables to manage solutions, agent communications, and conditions for stop criteria. Initialization of the search procedure. II: Constructionof starting configurations with initial levels for for the Construction of starting configurations with initial levels the free parameters. III: Agent-based configuration: application the search procedure Agent-based configuration: application of of the search procedure and exchange of best-so-far solutions among the agents, until stop criteria meet. IV: Organization of partialand global solutions. Report best-so-far partial and global solutions. Report best-so-far configuration and Pareto-optimal solutions. D. Monett Díaz Humboldt University of Berlin Doctoral dissertation, 25.02.2005

52. Why agents for configuring? Distributed execution The (metaheuristic) algorithms and the agents could be physically distributed over a network. Remote execution Local agents can interact with other agents situated on remote computers, thus allowing for the remote execution of algorithms, which do not need to be located where the users are. Cooperation Agents can decide whether to cooperate or not, as well as to ask for Cooperation, if needed, in order to solve the original configuration problem. Furthermore, they can cooperate by exchanging best-so-far obtained solutions. Autonomy & specialization Agents that interact with the users do not need to know how to configure algorithms, nor to solve them or to manage solutions (and vice versa), for example, in order to operate and to have control over their actions. D. Monett Díaz Humboldt University of Berlin Doctoral dissertation, 25.02.2005

53. Why agents … Exploration of the search space Subproblems are considered so as to cover the complete search space as well as possible. Exploitation of the search space It is done by studying a free parameter in detail. The more different parameter variations are considered, the wider the analysis and study of the related parameter. Incremental quality solution Solutions are improved by each agent when applying the configuration algorithm and solutions received from other agents can also improve the ones obtained so far. D. Monett Díaz Humboldt University of Berlin Doctoral dissertation, 25.02.2005

54. Fine-tuning (II) Example by using GA Factor in study: Population size, PopSize Levels: 50, 100, 150 (3 configurations) Number of runs for each configuration: NRuns 1 PopSize=50 1 PopSize=100 1 PopSize=150 + + + fixed factors fixed factors fixed factors . . . . . . . . . NRuns PopSize=50 NRuns PopSize=100 NRuns PopSize=150 + + + fixed factors fixed factors fixed factors  (Factori * Levelj * Runsk ) Which is the “most acceptable” solution? Which is the “best-so-far” configuration? D. Monett Díaz Humboldt University of Berlin Doctoral dissertation, 25.02.2005

55. Fine-tuning (III) tunable parameters (or factors) = controllable parameters = free parameters parameter tuning = fine-tuning = parameter setting = configuring = configuration process Configuration a specific setting or combination of free and fixed parameters Restrictions define the levels or different values that parameters may have D. Monett Díaz Humboldt University of Berlin Doctoral dissertation, 25.02.2005

56. Worth of a metaheuristic Pareto-optimum: Situation where it is not possible to improve (to decrease) the value of an objective function without deteriorating (increasing) that of at least one other Example of worth equation (using the weighted sum approach) worth(C )  w1   s2  w2   t2  w3   v2 where: m std. deviations m  x  xi 2 x i 1 i Normalization (e.g.): 2 x  i 1 , x 2 x variances m averages m max(  s2 ,  t2 ,  v2 ) D. Monett Díaz Humboldt University of Berlin Doctoral dissertation, 25.02.2005

57. Agent-based Config.: Algorithm Configuration ABRRHC(c, p) { // c : initial configuration from SCB agent // p : index of the parameter to fine-tune bestConfiguration = c; i = j = k = 0; do while (i < MaxExchanges and condition1) { do while (j < MaxTrials and condition2) { do while (k < MaxNeighbors and condition3) { nc = neighborConfiguration(c, p); evaluate(nc); if quality(nc) ≤ quality(bestConfiguration) then bestConfiguration = nc; k++; } c = bestNeighbor(); if isRestartAllowed then { rc = restartConfiguration(); evaluate(rc); if quality(rc) ≤ quality(bestConfiguration) then bestConfiguration = rc; c = rc; } j++; } if isExchangeAllowed then { ec = exchangeSolution(bestSolution); if quality(ec) ≤ quality(bestConfiguration) then bestConfiguration = ec; c = ec; } i++; } return bestConfiguration; } // end ABRRHC

58. +CARPS : Development & tools • JADE (Java Agent DEvelopment Framework) • +CARPS classes are Java classes. • +CARPS packages enclose agents, ontologies, and utilities, separately. • Classes & hierarchies will be presented as they are showed when using the BlueJ Java development tool. D. Monett Díaz Humboldt University of Berlin Doctoral dissertation, 25.02.2005

60. Config vocabulary and ontology D. Monett Díaz Humboldt University of Berlin Doctoral dissertation, 25.02.2005

61. InstStrategy vocabulary and ontology D. Monett Díaz Humboldt University of Berlin Doctoral dissertation, 25.02.2005

62. +CARPS Agents Types of agents UM User Mediators ISM Instantiation Strategy Managers SCB Starting Configuration Builders AC Algorithm Configurators AS Algorithm Solvers SM Solution Managers - Agent communication: relevant - Specialization and distributed information processing: relevant - Following the standards (e.g. FIPA specifications): important D. Monett Díaz Humboldt University of Berlin Doctoral dissertation, 25.02.2005

63. SCB agent : Example P  D1 , P2 , P3 1 s1b  randInit ,   b   s2  lowLevelInit ,  SCB1 ST1   b  s3  uppLevelInit , s b  avgInit   4  level11 , level12 , level13 , level14 C1 : level11 , P2 , P3  C3 : level13 , P2 , P3  C 2 : level12 , P2 , P3  C 4 : level14 , P2 , P3  D. Monett Díaz Humboldt University of Berlin Doctoral dissertation, 25.02.2005

64. Algorithm Configurators: IPs AC & AS agents help UM agents with the configuration process UM agents book their services • HBIP : Helper-Booker Interaction Protocol – HB Initiator Finite State Machine (AC & AS) – HP Responder Finite State Machine (UM) • HelperBookerProtoASResponder.java • HelperBookerProtoACResponder.java D. Monett Díaz Humboldt University of Berlin Doctoral dissertation, 25.02.2005

65. Helper-Booker-Protocol AS, AC UM Initiator Participant propose reject-proposal failure accept-proposal inform failure [dead- line1] confirm cancel [deadline2] D. Monett Díaz Humboldt University of Berlin Doctoral dissertation, 25.02.2005

66. Helper-Booker-Initiator-FSM SEARCH_DF DELAY 1 VERIFY_REGISTERED_AGENTS REMOVE_BOOKER 2 4 12 SND_PROPOSAL 7 SND_CANCEL 3 10 6 RCV_RESPONSE RCV_DATA 5 8 9 SND_RES_NOTIFICATION 11 TERMINATE_PROTOCOL MAKE_CLONE

67. Helper-Booker-Responder-FSM RCV_PROPOSAL 1 2 VERIFY_BOOKED_AGENTS 4 3 13 SND_REJECT_PROPOSAL SND_ACCEPT_PROPOSAL 5 8 6 SND_DATA 7 RCV_RESPONSE 10 9 11 12 INCREASE_BOOKERS SND_CANCEL VERIFY_TERMINATION 14 TERMINATE_PROTOCOL

68. Agents’ communications Most relevant IPs in +CARPS Engagement-Protocol AC AS Interaction Protocols Initiator Participant query-if Helper-Booker IP refuse Engagement IP inform [deadline1] Request IP confirm [deadline2] cancel - Which agents will solve the algorithms? - Among AC agents (configurators) and AS agents (solvers) - Particular case of the standard FIPA Query IP - Configurators are initiators of the EIPs - Solvers are responders D. Monett Díaz Humboldt University of Berlin Doctoral dissertation, 25.02.2005

69. Algorithm Configurators: IPs An engagement is a compromise, a contract, between two parts AS agents solve the algorithm being configured Engagement Interaction Protocol AC agents need AS agents in the configuration process D. Monett Díaz Humboldt University of Berlin Doctoral dissertation, 25.02.2005

70. Algorithm Configurators: IPs • HBIP : Helper-Booker Interaction Protocol – HB Initiator Finite State Machine (AC & AS) • HelperBookerProtoInitiator.java – HP Responder Finite State Machine (UM) • EIP : Engagement Interaction Protocol – E Initiator Finite State Machine (AC) • EngagementProtoInitiator.java – E Responder Finite State Machine (AS) • Request Interaction Protocol – Request Initiator Finite State Machine (AC) • RequestProtoInitiator.java D. Monett Díaz Humboldt University of Berlin Doctoral dissertation, 25.02.2005

71. Algorithm Configurators: IPs Engagement-Protocol AC AS Initiator Participant query-if refuse inform [deadline1] confirm cancel [deadline2] D. Monett Díaz Humboldt University of Berlin Doctoral dissertation, 25.02.2005

72. Algorithm Configurators: IPs Engagement-Initiator-FSM SELECT_SOLVER 2 SND_QUERY_IF REMOVE_SOLVER 1 4 6 5 RCV_RESPONSE SND_RES_NOTIFICATION 3 7 TERMINATE_PROTOCOL D. Monett Díaz Humboldt University of Berlin Doctoral dissertation, 25.02.2005

73. Engagement-Responder-FSM 1 RCV_QUERY_IF 2 VERIFY_CONDITIONS 10 4 3 SND_REFUSE SND_INFORM 5 6 RCV_RESPONSE 7 8 9 ENGAGE SND_CANCEL VERIFY_TERMINATION 11 TERMINATE_PROTOCOL D. Monett Díaz Humboldt University of Berlin Doctoral dissertation, 25.02.2005

74. Agents’ communications Most relevant IPs in +CARPS FIPA-Request-Protocol Initiator Participant Interaction Protocols request refuse Helper-Booker IP [refused] agree [agreed and Engagement IP notification necessary] Request IP failure [agreed] inform-result: inform - Solving the algorithms - Among AC agents (configurators) and AS agents (solvers) - FIPA Request IP - like - Configurators are initiators of the EIPs - Solvers are responders D. Monett Díaz Humboldt University of Berlin Doctoral dissertation, 25.02.2005

75. Agents’ communications Request-Initiator-FSM SND_REQUEST 8 1 RCV_RESPONSE 3 2 5 RCV_RES_NOTIFICATION UPDATE_CONTENT 4 6 7 9 RCV_DATA TERMINATE_PROTOCOL D. Monett Díaz Humboldt University of Berlin Doctoral dissertation, 25.02.2005

76. Solver-FSM PREPARE_DATA 1 VERIFY_REPETITIONS EXEC_ALGORITHM 5 4 3 2 VERIFY_SOLUTIONS UPDATE_COUNTER 6 7 PROCESS_RESULTS CONSTRUCT_SOLUTIONS UPDATE_VARIABLES PROCESS_ERR_CODE TERMINATE_SOLVER D. Monett Díaz Humboldt University of Berlin Doctoral dissertation, 25.02.2005

77. AC-AS communications paths to M and io files, par. names, ag. lists data configs AC AS M solutions results input files config AC AS M .exe singular solution output files D. Monett Díaz Humboldt University of Berlin Doctoral dissertation, 25.02.2005

78. +CARPS : Tree +carps src agent AC behaviours AS ISM Agents and their behaviors SCB SM UM metah copga Algorithms to configure evoes ontology agentList configuration Vocabularies and ontologies strategy userproblem util comm graphics gui Utilities io jExtra table D. Monett Díaz Humboldt University of Berlin Doctoral dissertation, 25.02.2005

79. <time.h> GA <windows.h> time.hpp time.cpp commonDef.hpp <math.h> random.hpp random.cpp <assert.h> <fstream.h> utility.hpp utility.cpp <iostream.h> gene.hpp gene.gpp <iomanip.h> chromosome.hpp chromosome.cpp objfunc.hpp objfunc.cpp population.hpp population.cpp ga.hpp ga.cpp <ctype.h> <stdlib.h> copga.hpp copga.cpp <string.h> mainGA.cpp D. Monett Díaz Humboldt University of Berlin Doctoral dissertation, 25.02.2005

80. Agents and containers AC1 AC2 DF AMS RMA UM1 SM1 ISM1 SCB1 Main container AS1 AS2 Container 1 Container 2 Platform 1 Network UM2 ISM2 SM2 SCB2 AS3 AS4 DF AMS RMA DF AMS RMA Main container Main container Platform 3 Platform 2

81. JADE: Sniffer agent

84. ES: c vs. quality theoretical value for the optimum: c = 1 and c = -1 60 5 50 4 40 3 quality quality 30 2 20 1 10 0 0 -9,97 -7,15 -4,77 -2,42 -0,73 1,70 3,90 6,63 9,21 -9,97 -7,15 -4,77 -2,42 -0,73 1,70 3,90 6,63 9,21 c c 1 0,004 0,8 0,003 quality 0,6 quality 0,002 0,4 0,001 0,2 0 0 -10,0 -8,00 -6,00 -4,00 -2,00 0,00 2,00 4,00 6,00 8,00 10,00 -9,97 -7,15 -4,77 -2,42 -0,73 1,70 3,90 6,63 9,21 0 c c D. Monett Díaz Humboldt University of Berlin Doctoral dissertation, 25.02.2005

85. 1prop&4n 10 8 6 4 2 0 c -2 -4 9,0E+03 -6 8,0E+03 -8 7,0E+03 -10 time (msec) 6,0E+03 0 2 4 6 8 10 5,0E+03 lambda 4,0E+03 3,0E+03 9,0E+04 2,0E+03 8,0E+04 1,0E+03 7,0E+04 0,0E+00 6,0E+04 0 5 10 15 20 25 30 35 40 45 func. eval. 5,0E+04 quality 4,0E+04 3,0E+04 2,0E+04 1,0E+04 0,0E+00 0 5 10 15 20 25 30 35 40 45 quality D. Monett Díaz Humboldt University of Berlin Doctoral dissertation, 25.02.2005

86. Testing engagements Elapsed time of the Engagement IPs - Proportion of AC agents per parameter to fine-tune = 2 - Intra-platform communication - AC agents run in a main container; AS agents, in a secondary one - Number of parameter to fine-tune = 2 ( and c) 25 20 4 AC agents & 4 AS agents time (sec) 15 10 Engagements: 5 AC1-AS2 (or AC1-AS4) AC2-AS3 0 1 2 3 4 AC3-AS1 AC 0,094 11,688 23,359 0,094 AC4-AS4 (or AC1-AS2) AS 23,875 0,359 12,11 0,344 D. Monett Díaz Humboldt University of Berlin Doctoral dissertation, 25.02.2005

87. Pareto-optimal singular solutions 60 50 40 Weight vector (w1, w2, w3) = (1.0, 0.0, 0.0) quality 30 20 10 0 The importance goes to the solution qualities 0 1000 2000 3000 4000 5000 6000 (when calculating the worth) time (msec) 60 50 40 Four best Pareto-optimal singular solutions according to their qualities quality 30 20 10 lambda c quality 0 5,15965245 0,87023895 3,98E-05 0 5000 10000 15000 20000 25000 30000 35000 4,86909981 -0,98344576 5,70E-05 func.eval. 4,4049138 -1,00838506 3,43E-04 4,83154784 -1,04477195 3,57E-04 Total of Pareto-optimal = 388 singular solutions D. Monett Díaz Humboldt University of Berlin Doctoral dissertation, 25.02.2005

88. (cont.) 1,2E+04 1,0E+04 time (msec) 8,0E+03 6,0E+03 4,0E+03 2,0E+03 4,0E+04 3,5E+04 0,0E+00 1 2 3,0E+04 time (msec) AC 10985 94 2,5E+04 AS 11266 141 2,0E+04 1,5E+04 1,0E+04 6,0E+04 5,0E+03 0,0E+00 5,0E+04 1 2 3 4 109 35843 11000 11015 time (msec) 4,0E+04 AC AS 11375 203 11422 36172 3,0E+04 2,0E+04 1,0E+04 0,0E+00 1 2 3 4 5 6 AC 156 57953 13500 125 93 156 AS 437 14687 469 438 469 59328 D. Monett Díaz Humboldt University of Berlin Doctoral dissertation, 25.02.2005

89. (cont.) 14 12 10 best quality 8 6 4 2 0 2,50E-03 4,00E-03 1 30 59 88 117 146 175 204 233 262 3,50E-03 2,00E-03 t 3,00E-03 1,50E-03 best worth 2,50E-03 8n 6n 4n 2n 1,00E-03 2,00E-03 1,50E-03 5,00E-04 1,00E-03 0,00E+00 5,00E-04 5,0E-04 -5,00E-04 0,00E+00 4,0E-04 1 30 59 88 117 146 175 204 233 262 best quality 3,0E-04 t 2n 4n 8n 6n 2,0E-04 1,0E-04 0,0E+00 1 30 59 88 117 146 175 204 233 262 t 4n 8n D. Monett Díaz Humboldt University of Berlin Doctoral dissertation, 25.02.2005

90. q9vp 1,8 1,6 r2 1,4 1,2 15 1 12 best fitness 2 2,5 3 3,5 4 9 r1 6 3 0,09 0 0,088 1 10 19 28 37 46 55 64 73 82 91 100 0,086 population best distance 0,084 0,082 current best-so-far 0,08 0,078 0,076 0,074 1 10 19 28 37 46 55 64 73 82 91 100 population current best-so-far D. Monett Díaz Humboldt University of Berlin Doctoral dissertation, 25.02.2005

91. 2n&repeat 1,4E+06 1,2E+06 1,0E+06 time (msec) 8,0E+05 6,0E+05 4,0E+05 2,0E+05 0,0E+00 2n-1 2n-2 2n-3 2n-4 2n-5 mean Solver 911399 798396 1136179 982935 992848 964351,4 ABRRHC 99601 87823 108009 112955 103573 102392,2 1,4E+06 1,2E+06 1,0E+06 time (msec) 8,0E+05 6,0E+05 4,0E+05 2,0E+05 0,0E+00 2n-1 2n-2 2n-3 2n-4 2n-5 Solver 911399 798396 1136179 982935 992848 ABRRHC 99601 87823 108009 112955 103573 D. Monett Díaz Humboldt University of Berlin Doctoral dissertation, 25.02.2005

92. (cont.) 2,5E+07 2,0E+07 time (msec) 1,5E+07 1,0E+07 5,0E+06 0,0E+00 1x 2x 3x Solver 2414233 7422779 16841628 ABRRHC 185971 1141846 3319888 60 50 40 time (min) 30 20 10 0 1x 2x 3x Solver 20,11860833 30,92824583 46,7823 ABRRHC 1,549758333 4,757691667 9,221911111 D. Monett Díaz Humboldt University of Berlin Doctoral dissertation, 25.02.2005

93. Related publications D. Monett (2004). Interaction Protocols for +CARPS Agents: Booking and Getting Engaged for Configuring. In G. Lindemann, H.-D. Burkhard, L. Czaja, A. Skowron, H. Schlingloff, Z. Suraj, editors, Proceedings of the Workshop Concurrency, Specification, and Programming CS&P'2004, volume 3: Multiagent Systems and Applications, pages 507-518, Caputh, Germany. Informatik-Bericht Nr. 170. (Also in Special Issue of Fundamenta Informaticae –to appear–) D. Monett (2004). Collaborative JADE Agents Enabling the Configuration of Algorithms. In D. Khadraoui, editor, Proceedings of the International Conference on Advances in Intelligent Systems - Theory and Applications, AISTA'2004, IEEE Computer Society, University of Canberra and CRP Henri Tudor, Luxembourg-Kirchberg, Luxembourg. D. Monett (2004). +CARPS: Configuration of Metaheuristics Based on Cooperative Agents. In Ch. Blum, A. Roli, M. Sampels, editors, Proceedings of the First International Workshop on Hybrid Metaheuristics, HM'2004, at the 16th European Conference on Artificial Intelligence, ECAI'2004, pages 115-125, Valencia, Spain. D. Monett, J.A. Méndez, G.A. Abraham, A. Gallardo, J. San Román (2002). An Evolutionary Approach to Reactivity Ratios Prediction. Macromol. Theory Simul., 11(5):525-532. Wiley-VCH Verlag GmbH, Weinheim, Germany. D. Monett (2001). On the automation of evolutionary techniques and their application to inverse problems from Chemical Kinetics. In C. Ryan, editor, Proceedings of the GECCO'01 Graduate Student Workshop, Genetic and Evolutionary Computation Conference, pages 429-432, San Francisco, California, USA. D. Monett (2003). Configuration of Metaheuristics: Overview and Theoretical Approach. In L. Czaja, editor, Proceedings of the Workshop Concurrency, Specification, and Programming CS&P'2003, volume 2, pages 353-364, Czarna, Poland. Zakład Graficzny UW, zam. 591/2003. D. Monett Díaz Humboldt University of Berlin Doctoral dissertation, 25.02.2005

94. Conclusions and Contributions +CARPS, agent-based approach to the configuration of algorithms including, but not limited to, metaheuristics is proposed +CARPS consists of different types of cooperative agents that support the autonomous configuration of metaheuristic algorithms and that follows FIPA specifications in a distributed fashion Ontologies, interaction protocols, agent behaviors, and other supporting classes conform the +CARPS infrastructure for the agent-based configuration Conception and development of new interaction protocols that are followed by the agents in order to cover communication requirements +CARPS needed in the domain of analysis I/O Layer Algorithm Configuration Layer Algorithm Solution Layer Communication Layer D. Monett Díaz Humboldt University of Berlin Doctoral dissertation, 25.02.2005

95. Conclusions and Contributions Theoretical description and formalization of the configuration problem D. Monett (2003). Configuration of Metaheuristics: Overview and Theoretical Approach. In Proceedings of the Workshop Concurrency, Specification, and Programming CS&P'2003, volume 2, Czarna, Poland. Implementation of a Random-Restart Hill-Climbing algorithm that some specialized agents apply to search for solutions during the configuration process +CARPS is also a framework in which monitoring of control factors of metaheuristics can be easily made At the same time, it can be seen as a powerful tool useful for conducting experiments when executing metaheuristic algorithms +CARPS I/O Layer Algorithm Configuration Layer Algorithm Solution Layer Communication Layer D. Monett Díaz Humboldt University of Berlin Doctoral dissertation, 25.02.2005

Agent-Based Configuration of (Metaheuristic) Algorithms - Doctoral dissertation

Agent-Based Configuration of (Metaheuristic) Algorithms - Doctoral dissertation

Recommended

More Related Content

Viewers also liked

Viewers also liked(20)

Similar to Agent-Based Configuration of (Metaheuristic) Algorithms - Doctoral dissertation

Similar to Agent-Based Configuration of (Metaheuristic) Algorithms - Doctoral dissertation(20)

More from Dagmar Monett

More from Dagmar Monett(15)

Recently uploaded

Recently uploaded(20)

Agent-Based Configuration of (Metaheuristic) Algorithms - Doctoral dissertation