Successfully reported this slideshow.
Upcoming SlideShare
×

# Covariance Matrix Adaptation Evolution Strategy - CMA-ES

3,444 views

Published on

Published in: Science
• Full Name
Comment goes here.

Are you sure you want to Yes No
• Be the first to comment

### Covariance Matrix Adaptation Evolution Strategy - CMA-ES

1. 1. Covariance Matrix Adaptation Evolution Strategy (CMA-ES) BY: OSAMA SALAH ELDIN UNDER SUPERVISION: PROF. MAGDA B. FAYEK 6/3/2016 CAIRO UNIVERSITY - COMPUTER ENGINEERING - 2015
2. 2. Outline oWhat is Optimization? oWhat is an Evolution Strategy? oStep-size Adaptation oCumulative step-size adaptation oCovariance Matrix Adaptation oApplication - Modeling 6/3/2016 CAIRO UNIVERSITY - COMPUTER ENGINEERING - 2015 2
3. 3. What is optimization? oOptimization is the minimization or the maximization of a function 6/3/2016 CAIRO UNIVERSITY - COMPUTER ENGINEERING - 2015 3 y=f(x) Global MinimumLocal Minimum xLocal Minimum
4. 4. What is optimization? oTry to solve these problems: 6/3/2016 CAIRO UNIVERSITY - COMPUTER ENGINEERING - 2015 4 x = 2 x3 – 8 = 0
5. 5. What is optimization? oTry to solve these problems: 6/3/2016 CAIRO UNIVERSITY - COMPUTER ENGINEERING - 2015 5 x=3, y=2 x2 + 3.y – 15 = 0
6. 6. What is optimization? oTry to solve these problems: 6/3/2016 CAIRO UNIVERSITY - COMPUTER ENGINEERING - 2015 6 x=3, y=2, z=2 x2 + y + 2.z – 15 = 0
7. 7. What is optimization? oTry to solve these problems: 6/3/2016 CAIRO UNIVERSITY - COMPUTER ENGINEERING - 2015 7
8. 8. What is optimization? oTry to solve these problems: 6/3/2016 CAIRO UNIVERSITY - COMPUTER ENGINEERING - 2015 8 Can one try all combinations?? This is not recommended
9. 9. What is optimization? 6/3/2016 CAIRO UNIVERSITY - COMPUTER ENGINEERING - 2015 9 Use an evolution strategy
10. 10. Outline oWhat is Optimization? oWhat is an Evolution Strategy? oStep-size Adaptation oCumulative step-size adaptation oCovariance Matrix Adaptation oApplication - Modeling 6/3/2016 CAIRO UNIVERSITY - COMPUTER ENGINEERING - 2015 10
11. 11. 6/3/2016 CAIRO UNIVERSITY - COMPUTER ENGINEERING - 2015 11 What is an Evolution Strategy?
12. 12. What is an Evolution Strategy? oIt is a technique that searches for the optimum solution in a search-space oEvolution Strategies belong to the family of Evolutionary Computation oEvolution strategy steps: 1. Generate a population of candidate solutions 2. Evaluate every individual in the population 3. Select parents from the fittest individuals 4. Reproduce offspring of the next generation (Recombination & mutation) 5. Repeat until a termination criterion is met 6/3/2016 CAIRO UNIVERSITY - COMPUTER ENGINEERING - 2015 12
13. 13. Evolution Strategies VS. Genetic Algorithms ES GA Initial Population Random mutations of the initial guess Random or seeded Evaluation Objective Function Fitness (Evaluation) Function Selection Truncation Selection Different methods Reproduction Recombination + Mutation Crossover + Mutation Termination Almost similar stop conditions 6/3/2016 CAIRO UNIVERSITY - COMPUTER ENGINEERING - 2015 13
14. 14. What is an Evolution Strategy? - Example 1. Generate a population of candidate solutions 6/3/2016 CAIRO UNIVERSITY - COMPUTER ENGINEERING - 2015 14 y=f(x) x
15. 15. fitness What is an Evolution Strategy? - Example 2. Evaluate every individual in the population 6/3/2016 CAIRO UNIVERSITY - COMPUTER ENGINEERING - 2015 15 y=f(x) x
16. 16. fitness What is an Evolution Strategy? - Example 3. Select parents from the fittest individuals 6/3/2016 CAIRO UNIVERSITY - COMPUTER ENGINEERING - 2015 16 y=f(x) x
17. 17. What is an Evolution Strategy? - Example 4. Reproduce offspring of the next generation (Recombination & mutation) 6/3/2016 CAIRO UNIVERSITY - COMPUTER ENGINEERING - 2015 17 y=f(x) x
18. 18. What is an Evolution Strategy? - Example 5. Repeat until a termination criterion is met 6/3/2016 CAIRO UNIVERSITY - COMPUTER ENGINEERING - 2015 18 y=f(x) xEvaluate & Select
19. 19. What is an Evolution Strategy? - Example 5. Repeat until a termination criterion is met 6/3/2016 CAIRO UNIVERSITY - COMPUTER ENGINEERING - 2015 19 y=f(x) xEvaluate & SelectReproduce
20. 20. What is an Evolution Strategy? - Example 5. Repeat until a termination criterion is met 6/3/2016 CAIRO UNIVERSITY - COMPUTER ENGINEERING - 2015 20 y=f(x) x Optimum Solution Evaluate – Select – ReproduceReproduceTerminate
21. 21. The Basic Evolution Strategy oThe basic evolution strategy is defined by: (µ/ρ, λ)-ES and (µ/ρ+ λ)-ES Where: µ The number of selected individuals per generation ρ The number of parents (selected from µ) involved in recombination (≤ µ) λ The number of individuals per generation (population size) , Comma Selection  µ parents are selected from the λ individuals + Plus Selection  µ parents are selected from the λ individuals + the current ρ parents 6/3/2016 CAIRO UNIVERSITY - COMPUTER ENGINEERING - 2015 21
22. 22. The Basic Evolution Strategy - Example (10/6 , 50)-ES Select the fittest 10 individuals from the 50 individuals of the current population, and select 6 random ones from them. Recombine these 6 parents to generate 50 new offspring (10/6 + 50)-ES Select the fittest 10 individuals from the 50 individuals of the current population along with their 6 parents, and select 6 random ones from them all (from the 56). Recombine these 6 parents to generate 50 new offspring 6/3/2016 CAIRO UNIVERSITY - COMPUTER ENGINEERING - 2015 22
23. 23. The structure of an Individual Object Parameter Vector (Y) Strategy Parameter Vector (S) Individual’s Fitness (F) 6/3/2016 CAIRO UNIVERSITY - COMPUTER ENGINEERING - 2015 23 Y The candidate solution of the problem (e.g. (x, y) point) S The parameters used by the strategy (e.g. mutation strength) F The fitness of the candidate solution y as measured by the fitness function (i.e. the value of the objective function) Y = {x1, x2, z}
24. 24. The structure of an Individual Object Parameter Vector (Y) Strategy Parameter Vector (S) Individual’s Fitness (F) 6/3/2016 CAIRO UNIVERSITY - COMPUTER ENGINEERING - 2015 24 • Evolution strategies search for the optimum: 1. Solution: The highest fitness 2. Strategy Parameters: The fastest improvement Two search spaces
25. 25. ES Steps 6/3/2016 CAIRO UNIVERSITY - COMPUTER ENGINEERING - 2015 25 1- Initial Solution
26. 26. 2 - Initial Population ES Steps 6/3/2016 CAIRO UNIVERSITY - COMPUTER ENGINEERING - 2015 26 1- Initial Solution
27. 27. 2 - Initial Population ES Steps 6/3/2016 CAIRO UNIVERSITY - COMPUTER ENGINEERING - 2015 27 3 - Evaluation 1- Initial Solution
28. 28. 3 - Evaluation4 - Selection ES Steps 6/3/2016 CAIRO UNIVERSITY - COMPUTER ENGINEERING - 2015 28 µ = 4 ρ=1 2 - Initial Population 1- Initial Solution
29. 29. 5 - Reproduction ES Steps 6/3/2016 CAIRO UNIVERSITY - COMPUTER ENGINEERING - 2015 29 2 - Initial Population 1- Initial Solution 3 - Evaluation 4 - Selection
30. 30. 5 - Reproduction ES Steps 6/3/2016 CAIRO UNIVERSITY - COMPUTER ENGINEERING - 2015 30 2 - Initial Population 1- Initial Solution 3 - Evaluation 4 - Selection
31. 31. ES Steps 6/3/2016 CAIRO UNIVERSITY - COMPUTER ENGINEERING - 2015 31 2 - Initial Population 1- Initial Solution 3 - Evaluation 4 - Selection 5 - Reproduction
32. 32. ES Steps 6/3/2016 CAIRO UNIVERSITY - COMPUTER ENGINEERING - 2015 32 2 - Initial Population 1- Initial Solution 3 - Evaluation 4 - Selection 5 - Reproduction
33. 33. 6 - Termination ES Steps 6/3/2016 CAIRO UNIVERSITY - COMPUTER ENGINEERING - 2015 33 2 - Initial Population 1- Initial Solution 3 - Evaluation 4 - Selection Optimum Solution 5 - Reproduction
34. 34. The Basic Evolution Strategy 6/3/2016 CAIRO UNIVERSITY - COMPUTER ENGINEERING - 2015 34 2 - Initial Population 1- Initial Solution 3 - Evaluation 4 - Selection • An initial guess, should be as close as possible to the expected solution 6 - Termination 5 - Reproduction
35. 35. The Basic Evolution Strategy 6/3/2016 CAIRO UNIVERSITY - COMPUTER ENGINEERING - 2015 35 2 - Initial Population 1- Initial Solution 3 - Evaluation 4 - Selection • The intial population is generated by mutating the initial solution 6 - Termination 5 - Reproduction
36. 36. The Basic Evolution Strategy 6/3/2016 CAIRO UNIVERSITY - COMPUTER ENGINEERING - 2015 36 2 - Initial Population 1- Initial Solution 3 - Evaluation 4 - Selection • Every individual is evaluated by the objective function 6 - TerminationBest Fitness = 0 5 - Reproduction
37. 37. The Basic Evolution Strategy 6/3/2016 CAIRO UNIVERSITY - COMPUTER ENGINEERING - 2015 37 2 - Initial Population 1- Initial Solution 3 - Evaluation 4 - Selection • Truncation Selection is used 6 - Termination 5 - Reproduction Select the fittest µ individuals Drop the other individuals
38. 38. The Basic Evolution Strategy 6/3/2016 CAIRO UNIVERSITY - COMPUTER ENGINEERING - 2015 38 2 - Initial Population 1- Initial Solution 3 - Evaluation 4 - Selection 5 - Reproduction 6 - Termination Recombination Reproduction Mutation Combining two or more parents to produce a mean for the new generation Adding normally- distributed random vectors to the new mean
39. 39. The Basic Evolution Strategy 6/3/2016 CAIRO UNIVERSITY - COMPUTER ENGINEERING - 2015 39 2 - Initial Population 1- Initial Solution 3 - Evaluation 4 - Selection 5 - Reproduction 6 - Termination Recombination S1 F11 3 S2 F24 6 Solution Strategy Parameters S32.5 4.5 A simple recombination is taking the average P1 P2 Fitness To be calculated
40. 40. The Basic Evolution Strategy 6/3/2016 CAIRO UNIVERSITY - COMPUTER ENGINEERING - 2015 40 2 - Initial Population 1- Initial Solution 3 - Evaluation 4 - Selection 5 - Reproduction 6 - Termination Recombination ρ = 2 ρ = 4
41. 41. The Basic Evolution Strategy 6/3/2016 CAIRO UNIVERSITY - COMPUTER ENGINEERING - 2015 41 2 - Initial Population 1- Initial Solution 3 - Evaluation 4 - Selection 5 - Reproduction 6 - Termination Recombination Mutation Reproduction Combining two or more parents to produce a mean for the new generation Adding normally- distributed random vectors to the new mean
42. 42. The Basic Evolution Strategy 6/3/2016 CAIRO UNIVERSITY - COMPUTER ENGINEERING - 2015 42 2 - Initial Population 1- Initial Solution 3 - Evaluation 4 - Selection 5.5 8.0Parent RX1 RY1 Generate λ normally-distributed random vectors RX2 RY2 RX3 RY3 5.5 + RX1 8.0 + RY1 5.5 + RX2 8.0 + RY2 5.5 + RX3 8.0 + RY3 Add each of the λ mutating vectors to the initial solution 6 - Termination 5 - Reproduction Mutation Recombination
43. 43. The Basic Evolution Strategy 6/3/2016 CAIRO UNIVERSITY - COMPUTER ENGINEERING - 2015 43 2 - Initial Population 1- Initial Solution 3 - Evaluation 4 - Selection 5.5 8.0 RX1 RY1 RX2 RY2 RX3 RY3 RX1 RY1 5.5 + RX1 8.0 + RY1 5.5 + RX2 8.0 + RY2 5.5 + RX3 8.0 + RY3 RX2 RY2 RX3 RY3 5 - Reproduction Mutation 6 - Termination Recombination (5.5, 8.0)
44. 44. The Basic Evolution Strategy 6/3/2016 CAIRO UNIVERSITY - COMPUTER ENGINEERING - 2015 44 2 - Initial Population 1- Initial Solution 3 - Evaluation 4 - Selection 5.5 8.0 RX1 RY1 RX2 RY2 RX3 RY3 5.5 + RX1 8.0 + RY1 5.5 + RX2 8.0 + RY2 5.5 + RX3 8.0 + RY3 RX1 RY1 RX2 RY2 RX3 RY3 5 - Reproduction 6 - Termination Recombination Mutation
45. 45. 6 - Termination Mutation vectors are normally – distributed around their parent Recombination 5 - Reproduction The Basic Evolution Strategy 6/3/2016 CAIRO UNIVERSITY - COMPUTER ENGINEERING - 2015 45 2 - Initial Population 1- Initial Solution 3 - Evaluation 4 - Selection Repeat Mutation
46. 46. Outline oWhat is Optimization? oWhat is an Evolution Strategy? oStep-size Adaptation oCumulative step-size adaptation oCovariance Matrix Adaptation oApplication - Modeling 6/3/2016 CAIRO UNIVERSITY - COMPUTER ENGINEERING - 2015 46
47. 47. Step-size Adaptation (σSA-ES) 6/3/2016 CAIRO UNIVERSITY - COMPUTER ENGINEERING - 2015 47 Many small steps
48. 48. Step-size Adaptation (σSA-ES) 6/3/2016 CAIRO UNIVERSITY - COMPUTER ENGINEERING - 2015 48
49. 49. Step-size Adaptation (σSA-ES) 6/3/2016 CAIRO UNIVERSITY - COMPUTER ENGINEERING - 2015 49 These gaps are unrealistic Why ??
50. 50. Step-size Adaptation (σSA-ES) 6/3/2016 CAIRO UNIVERSITY - COMPUTER ENGINEERING - 2015 50 The parent of a generation is an individual in the previous generation
51. 51. Step-size Adaptation (σSA-ES) 6/3/2016 CAIRO UNIVERSITY - COMPUTER ENGINEERING - 2015 51
52. 52. Step-size Adaptation (σSA-ES) 6/3/2016 CAIRO UNIVERSITY - COMPUTER ENGINEERING - 2015 52
53. 53. 6 - Termination Recombination 5 - Reproduction The Basic Evolution Strategy - Revisited 6/3/2016 CAIRO UNIVERSITY - COMPUTER ENGINEERING - 2015 53 2 - Initial Population 1- Initial Solution 3 - Evaluation 4 - Selection Mutation Mutation
54. 54. The Gaussian (Normal) Distribution 6/3/2016 CAIRO UNIVERSITY - COMPUTER ENGINEERING - 2015 54 N ( µ , σ2) Mean (Standard Deviation)2 = Variance ≡ µ + σ.N (0, 1) µ + σ.N (0, I) Multivariate
55. 55. 6 - Termination Recombination 5 - Reproduction The Basic Evolution Strategy - Revisited 6/3/2016 CAIRO UNIVERSITY - COMPUTER ENGINEERING - 2015 55 2 - Initial Population 1- Initial Solution 3 - Evaluation 4 - Selection Mutation Mutation Step-size
56. 56. The Gaussian (Normal) Distribution 6/3/2016 CAIRO UNIVERSITY - COMPUTER ENGINEERING - 2015 56 σ=1.0σ=1.5σ=2.0σ=2.5
57. 57. Step-size Adaptation (σSA-ES) 6/3/2016 CAIRO UNIVERSITY - COMPUTER ENGINEERING - 2015 57 Object Parameter Vector (Y) Individual’s Fitness (F) Strategy Parameter Vector (S) σ n = length(Y): Problem dimension
58. 58. Outline oWhat is Optimization? oWhat is an Evolution Strategy? oStep-size Adaptation oCumulative step-size adaptation oCovariance Matrix Adaptation oApplication - Modeling 6/3/2016 CAIRO UNIVERSITY - COMPUTER ENGINEERING - 2015 58
59. 59. Cumulative Step-size Adaptation (CSA) oStep-size Adaptation 6/3/2016 CAIRO UNIVERSITY - COMPUTER ENGINEERING - 2015 59 Increase step size Decrease step size
60. 60. Cumulative Step-size Adaptation (CSA) 6/3/2016 CAIRO UNIVERSITY - COMPUTER ENGINEERING - 2015 60 1. Calculate the average 𝑍𝑡 of the fittest µ solutions 2. Calculate the cumulative path Pc at generation t The parameter c is called the cumulation parameter, it determines how rapidly the information stored in Pct fades. The typical value of c is between 1/n and 1/
61. 61. Cumulative Step-size Adaptation (CSA) 3. Update the mutation strength (i.e. step-size) 6/3/2016 CAIRO UNIVERSITY - COMPUTER ENGINEERING - 2015 61 The damping parameter dσ determines how much the step-size can change. (Normally, it is set to 1) Where ||𝑋||‖ is the Euclidean norm of the vector =
62. 62. Cumulative Step-size Adaptation (CSA) 6/3/2016 CAIRO UNIVERSITY - COMPUTER ENGINEERING - 2015 62 Object Parameter Vector (Y) σ Individual’s Fitness (F)
63. 63. Outline oWhat is Optimization? oWhat is an Evolution Strategy? oStep-size Adaptation oCumulative step-size adaptation oCovariance Matrix Adaptation oApplication - Modeling 6/3/2016 CAIRO UNIVERSITY - COMPUTER ENGINEERING - 2015 63
64. 64. Promising individuals Covariance-Matrix Adaptation (CMA) 6/3/2016 CAIRO UNIVERSITY - COMPUTER ENGINEERING - 2015 64
65. 65. Covariance-Matrix Adaptation (CMA) 6/3/2016 CAIRO UNIVERSITY - COMPUTER ENGINEERING - 2015 65
66. 66. Covariance-Matrix Adaptation (CMA) 6/3/2016 CAIRO UNIVERSITY - COMPUTER ENGINEERING - 2015 66
67. 67. Covariance-Matrix Adaptation (CMA) 6/3/2016 CAIRO UNIVERSITY - COMPUTER ENGINEERING - 2015 67
68. 68. Covariance-Matrix Adaptation (CMA) 6/3/2016 CAIRO UNIVERSITY - COMPUTER ENGINEERING - 2015 68 N (0, I) N (0, C) How?? Adapt the Covariance MatrixIdentity Matrix
69. 69. Covariance-Matrix 6/3/2016 CAIRO UNIVERSITY - COMPUTER ENGINEERING - 2015 69
70. 70. Covariance-Matrix 6/3/2016 CAIRO UNIVERSITY - COMPUTER ENGINEERING - 2015 70
71. 71. Covariance-Matrix 6/3/2016 CAIRO UNIVERSITY - COMPUTER ENGINEERING - 2015 71
72. 72. Covariance-Matrix 6/3/2016 CAIRO UNIVERSITY - COMPUTER ENGINEERING - 2015 72
73. 73. Covariance-Matrix 6/3/2016 CAIRO UNIVERSITY - COMPUTER ENGINEERING - 2015 73
74. 74. Covariance-Matrix Adaptation (CMA) oTo which direction should the population be directed? 6/3/2016 CAIRO UNIVERSITY - COMPUTER ENGINEERING - 2015 74 VarianceCovariance
75. 75. oVariance is a measure of how far a variable changes away from its mean Variance 6/3/2016 CAIRO UNIVERSITY - COMPUTER ENGINEERING - 2015 75 , 𝑋 is the mean of the samples of X
76. 76. 0 2 4 6 8 10 12 0 2 4 6 8 10 12 Variance 6/3/2016 CAIRO UNIVERSITY - COMPUTER ENGINEERING - 2015 76 High Variance Low Variance mean mean
77. 77. oCovariance is a measure of how two variables change together Covariance 6/3/2016 CAIRO UNIVERSITY - COMPUTER ENGINEERING - 2015 77
78. 78. Covariance 6/3/2016 CAIRO UNIVERSITY - COMPUTER ENGINEERING - 2015 78 (a) (c) Cov(a, c) = -3.0091 (b) Cov(a, b) = 8.3909
79. 79. Covariance-Matrix oIt is a matrix whose (i, j) element is the covariance between the ith and the jth variables 6/3/2016 CAIRO UNIVERSITY - COMPUTER ENGINEERING - 2015 79
80. 80. Covariance-Matrix Adaptation (CMA) oTo which direction should the population be directed? 6/3/2016 CAIRO UNIVERSITY - COMPUTER ENGINEERING - 2015 80 Variance=σ2Covariance
81. 81. Principal Component 6/3/2016 CAIRO UNIVERSITY - COMPUTER ENGINEERING - 2015 81 oCMA-ES performs a type of Principal Component Analysis (PCA) oPrincipal Component: The principal variable (component) is equivalent to the principal player: 1. High Variance 2. Low Covariance with other components  Distinct, or very special
82. 82. Covariance-Matrix Adaptation (CMA) 6/3/2016 CAIRO UNIVERSITY - COMPUTER ENGINEERING - 2015 82 oTo which direction should the population be directed?  Towards the principal component
83. 83. Covariance-Matrix Adaptation (CMA) 6/3/2016 CAIRO UNIVERSITY - COMPUTER ENGINEERING - 2015 83 • The optimum solution is (5, 50)  A practical run of CMA-ES x • The population moves faster towards the direction of the second component (50) • The initial guess is (0, 0)
84. 84. Covariance-Matrix Adaptation (CMA) 6/3/2016 CAIRO UNIVERSITY - COMPUTER ENGINEERING - 2015 84 optimum (5, 50) Initial (0, 0) generation Variance (σ2) Step-size
85. 85. CMA-ES (Steps)-1 oInitial Values ◦ C = I (n x n Identity Matrix) ◦ An initial guess m (n x 1 mean of the initial population) ◦ An initial step size (n x 1 standard-deviation matrix) 1. Generate λ offspring by mutating the mean m: 2. Evaluate the λ offspring 3. Sort the offspring by fitness so that: 6/3/2016 CAIRO UNIVERSITY - COMPUTER ENGINEERING - 2015 85 Fittest Individual
86. 86. CMA-ES (Steps)-2 4. Update the mean m of the population 6/3/2016 CAIRO UNIVERSITY - COMPUTER ENGINEERING - 2015 86  Weighted average The constants wi are selected such that: µ is the number of parents
87. 87. CMA-ES (Steps)-3 5. Update step-size cumulation path 𝑃 𝜎 : , where:  The random vector that generated the individual xi:λ ◦ cσ : Decay rate for evolution path for step-size σ (≈ 4/n) 6/3/2016 CAIRO UNIVERSITY - COMPUTER ENGINEERING - 2015 87
88. 88. CMA-ES (Steps)-4 6. Update the covariance-matrix cumulation path Pc ∈ ℝ(nx1): cc: Decay rate for evolution path of C 7. Update the step-size σ: Where ||X|| is the Euclidean norm of the vector X(m) = 6/3/2016 CAIRO UNIVERSITY - COMPUTER ENGINEERING - 2015 88
89. 89. CMA-ES (Steps)-5 8. Update the covariance matrix C: c1: Learning rate for rank-one update of C(≈ 2/n2) cµ: Learning rate for rank-µ update of C (≈ µw/n2) 6/3/2016 CAIRO UNIVERSITY - COMPUTER ENGINEERING - 2015 89 Repeat the previous steps until a satisfying solution is found or a maximum number of generations is exceeded or no significant improvement is achieved
90. 90. Advantages of CMA-ES oCMA-ES can outperform other strategies in the following cases: ◦ Non-separable problems (the parameters of the objective function are dependent) ◦ The derivative of the objective function is not available ◦ High dimension problems (n is large) ◦ Very large search spaces 6/3/2016 CAIRO UNIVERSITY - COMPUTER ENGINEERING - 2015 90
91. 91. CMA-ES Limitations oCMA-ES can be outperformed by other strategies in the following cases: ◦ Partly separable problems (i.e. optimization of n-dimension objective function can be divided into a series of n optimizations of every single parameter) ◦ The derivative of the objective function is easily available (Gradient Descend / Ascend) ◦ Small dimension problems ◦ Problems that can be solved using a relatively small number of function evaluations (e.g. < 10n evaluations) 6/3/2016 CAIRO UNIVERSITY - COMPUTER ENGINEERING - 2015 91
92. 92. Outline oWhat is Optimization? oWhat is an Evolution Strategy? oStep-size Adaptation oCumulative step-size adaptation oCovariance Matrix Adaptation oApplication - Modeling 6/3/2016 CAIRO UNIVERSITY - COMPUTER ENGINEERING - 2015 92
93. 93. Application - Modeling 6/3/2016 CAIRO UNIVERSITY - COMPUTER ENGINEERING - 2015 93 f(x)X y 2. Guess a model f(x) = a.x2 + b.x + c 1. Collect Samples x1 x2 x3 . . xn y1 y2 y3 . . yn 3. Optimize the model Find the optimum values of {a, b, c}
94. 94. Application – Modeling in Robocode Motion Model 6/3/2016 CAIRO UNIVERSITY - COMPUTER ENGINEERING - 2015 94 Find a model for this path
95. 95. Application – Modeling in Robocode Motion Model – Steps 1. Collect Samples: The (x, y) location of the enemy 2. Guess the model (using GA) 6/3/2016 CAIRO UNIVERSITY - COMPUTER ENGINEERING - 2015 95 3. Optimize the model
96. 96. Application – Modeling in Robocode Motion Model – Observations oDifferent models give different human-like behaviors 6/3/2016 CAIRO UNIVERSITY - COMPUTER ENGINEERING - 2015 96 Careless Reckless Tricky
97. 97. Using the Source Code oThe source code (m-file for MATLAB) for CMA-ES (C, C++, Java, Fortran, Python, R, Scilab, Matlab / Octave) is available at: https://www.lri.fr/~hansen/cmaes_inmatlab.html ◦ purecmaes.m: Simple implementation ◦ cmaes.m: Production Code 1. Specify the initial values of the parameters (step-size, covariance matrix, initial guess, population size … etc.) 2. Define your objective function 6/3/2016 CAIRO UNIVERSITY - COMPUTER ENGINEERING - 2015 97 function f=obj_func(x) f = (calculate the error here) % e.g. f = x(1)^3 – 8; Matlab /Octave
98. 98. 3. Call the function Matlab /Octave Using the Source Code 6/3/2016 CAIRO UNIVERSITY - COMPUTER ENGINEERING - 2015 98 >> function_mfile( parameters)
99. 99. Questions 6/3/2016 CAIRO UNIVERSITY - COMPUTER ENGINEERING - 2015 99 osama.salah.eg@ieee.org
100. 100. Thanks 6/3/2016 CAIRO UNIVERSITY - COMPUTER ENGINEERING - 2015 100