Algorithm Portfolios for Noisy Optimization: Compare Solvers Early (LION8)
•
1 like•234 views
"Algorithm Portfolios for Noisy Optimization: Compare Solvers Early". Marie-Liesse Cauwet, Jialin Liu and Olivier Teytaud. The 8th Learning and Intelligent OptimizatioN Conference (LION8), 2014.
Recommended
Recommended
More Related Content
Similar to Algorithm Portfolios for Noisy Optimization: Compare Solvers Early (LION8)
Similar to Algorithm Portfolios for Noisy Optimization: Compare Solvers Early (LION8) (20)
Recently uploaded
Recently uploaded (20)
Algorithm Portfolios for Noisy Optimization: Compare Solvers Early (LION8)
- 1. Algorithm Portfolios for Noisy Optimization: Compare Solvers Early Marie-Liesse CAUWET, Jialin LIU, Olivier TEYTAUD Outline Black-box Noisy Optimization Framework Algorithm Portfolios Noisy Optimization Algorithms (NOAs) Experiments Conclusions References Algorithm Portfolios for Noisy Optimization: Compare Solvers Early TAO Team INRIA Saclay-LRI-CNRS, Univ. Paris-Sud 91190 Gif-sur-Yvette, France Marie-Liesse CAUWET Jialin LIU Olivier TEYTAUD February 2014
- 2. Algorithm Portfolios for Noisy Optimization: Compare Solvers Early Marie-Liesse CAUWET, Jialin LIU, Olivier TEYTAUD Outline Black-box Noisy Optimization Framework Algorithm Portfolios Noisy Optimization Algorithms (NOAs) Experiments Conclusions References Introduction Usually: Portfolio of algorithms ! Combinatorial Optimization (C.O.) New: Portfolio of algorithms ! Noisy Optimization (N.O.)
- 3. Algorithm Portfolios for Noisy Optimization: Compare Solvers Early Marie-Liesse CAUWET, Jialin LIU, Olivier TEYTAUD Outline Black-box Noisy Optimization Framework Algorithm Portfolios Noisy Optimization Algorithms (NOAs) Experiments Conclusions References 1 Black-box Noisy Optimization Framework 2 Algorithm Portfolios 3 Noisy Optimization Algorithms (NOAs) 4 Experiments 5 Conclusions 6 References
- 4. Algorithm Portfolios for Noisy Optimization: Compare Solvers Early Marie-Liesse CAUWET, Jialin LIU, Olivier TEYTAUD Outline Black-box Noisy Optimization Framework Algorithm Portfolios Noisy Optimization Algorithms (NOAs) Experiments Conclusions References Black-box1 Noisy Optimization Framework Let f = f (x,!) from a domain D 2 Rd to R with ! random variable. We wish to find: argmin x E!f (x,!) We have access to independent evaluations of f . Notation: f (x) refers to f (x,!). 1Black-box: we have no knowledge about the noise.
- 5. Algorithm Portfolios for Noisy Optimization: Compare Solvers Early Marie-Liesse CAUWET, Jialin LIU, Olivier TEYTAUD Outline Black-box Noisy Optimization Framework Algorithm Portfolios Noisy Optimization Algorithms (NOAs) Experiments Conclusions References Black-box Noisy Optimization Framework Stochastic problem; 2Image from http://ethanclements.blogspot.fr/2010/12/postmodernism-essay-question. html
- 6. Algorithm Portfolios for Noisy Optimization: Compare Solvers Early Marie-Liesse CAUWET, Jialin LIU, Olivier TEYTAUD Outline Black-box Noisy Optimization Framework Algorithm Portfolios Noisy Optimization Algorithms (NOAs) Experiments Conclusions References Black-box Noisy Optimization Framework Stochastic problem; limited budget (here: total number of evaluations); 2Image from http://ethanclements.blogspot.fr/2010/12/postmodernism-essay-question. html
- 7. Algorithm Portfolios for Noisy Optimization: Compare Solvers Early Marie-Liesse CAUWET, Jialin LIU, Olivier TEYTAUD Outline Black-box Noisy Optimization Framework Algorithm Portfolios Noisy Optimization Algorithms (NOAs) Experiments Conclusions References Black-box Noisy Optimization Framework Stochastic problem; limited budget (here: total number of evaluations); target: anytime convergence to the optimum; 2Image from http://ethanclements.blogspot.fr/2010/12/postmodernism-essay-question. html
- 8. Algorithm Portfolios for Noisy Optimization: Compare Solvers Early Marie-Liesse CAUWET, Jialin LIU, Olivier TEYTAUD Outline Black-box Noisy Optimization Framework Algorithm Portfolios Noisy Optimization Algorithms (NOAs) Experiments Conclusions References Black-box Noisy Optimization Framework Stochastic problem; limited budget (here: total number of evaluations); target: anytime convergence to the optimum; black-box. 2 How to choose a suitable solver/optimizer? 2Image from http://ethanclements.blogspot.fr/2010/12/postmodernism-essay-question. html
- 9. Algorithm Portfolios for Noisy Optimization: Compare Solvers Early Marie-Liesse CAUWET, Jialin LIU, Olivier TEYTAUD Outline Black-box Noisy Optimization Framework Algorithm Portfolios Noisy Optimization Algorithms (NOAs) Experiments Conclusions References Black-box Noisy Optimization Framework Stochastic problem; limited budget (here: total number of evaluations); target: anytime convergence to the optimum; black-box. Algorithm Portfolios
- 10. Algorithm Portfolios for Noisy Optimization: Compare Solvers Early Marie-Liesse CAUWET, Jialin LIU, Olivier TEYTAUD Outline Black-box Noisy Optimization Framework Algorithm Portfolios Noisy Optimization Algorithms (NOAs) Experiments Conclusions References Algorithm Portfolios A finite number of given noisy optimization solvers, “orthogonal”; distribution of budget; information sharing.
- 11. Algorithm Portfolios for Noisy Optimization: Compare Solvers Early Marie-Liesse CAUWET, Jialin LIU, Olivier TEYTAUD Outline Black-box Noisy Optimization Framework Algorithm Portfolios Noisy Optimization Algorithms (NOAs) Experiments Conclusions References Algorithm Portfolios A finite number of given noisy optimization solvers, “orthogonal”; distribution of budget; information sharing. ! Performs almost as well as the best solver
- 12. Algorithm Portfolios for Noisy Optimization: Compare Solvers Early Marie-Liesse CAUWET, Jialin LIU, Olivier TEYTAUD Outline Black-box Noisy Optimization Framework Algorithm Portfolios Noisy Optimization Algorithms (NOAs) Experiments Conclusions References Algorithm Portfolios Algorithm 1 Noisy Optimization Portfolio Algorithm (NOPA).
- 13. Algorithm Portfolios for Noisy Optimization: Compare Solvers Early Marie-Liesse CAUWET, Jialin LIU, Olivier TEYTAUD Outline Black-box Noisy Optimization Framework Algorithm Portfolios Noisy Optimization Algorithms (NOAs) Experiments Conclusions References Algorithm Portfolios Algorithm 1 Noisy Optimization Portfolio Algorithm (NOPA). 1: Parameters: a dimension d 2 N⇤ 2: Initialization: initialize a portfolio {S1, . . . , SM} containing M solvers 3: m, n 1
- 14. Algorithm Portfolios for Noisy Optimization: Compare Solvers Early Marie-Liesse CAUWET, Jialin LIU, Olivier TEYTAUD Outline Black-box Noisy Optimization Framework Algorithm Portfolios Noisy Optimization Algorithms (NOAs) Experiments Conclusions References Algorithm Portfolios Algorithm 1 Noisy Optimization Portfolio Algorithm (NOPA). 1: Parameters: a dimension d 2 N⇤ 2: Initialization: initialize a portfolio {S1, . . . , SM} containing M solvers 3: m, n 1 4: while (true) do 5: for i = 1 to M do I Fair budget distribution 6: Apply an iteration of solver Si until it has received at least n data samples 7: xi ,n the current recommendation by solver Si 8: end for
- 15. Algorithm Portfolios for Noisy Optimization: Compare Solvers Early Marie-Liesse CAUWET, Jialin LIU, Olivier TEYTAUD Outline Black-box Noisy Optimization Framework Algorithm Portfolios Noisy Optimization Algorithms (NOAs) Experiments Conclusions References Algorithm Portfolios Algorithm 1 Noisy Optimization Portfolio Algorithm (NOPA). 1: Parameters: a dimension d 2 N⇤ 2: Initialization: initialize a portfolio {S1, . . . , SM} containing M solvers 3: m, n 1 4: while (true) do 5: for i = 1 to M do I Fair budget distribution 6: Apply an iteration of solver Si until it has received at least n data samples 7: xi ,n the current recommendation by solver Si 8: end for 9: if n = rm then I Periodically we compare 10: for i = 1 to M do 11: Perform sm evaluations of the (stochastic) reward R(xi ,kn ) 12: yi the average reward 13: end for
- 16. Algorithm Portfolios for Noisy Optimization: Compare Solvers Early Marie-Liesse CAUWET, Jialin LIU, Olivier TEYTAUD Outline Black-box Noisy Optimization Framework Algorithm Portfolios Noisy Optimization Algorithms (NOAs) Experiments Conclusions References Algorithm Portfolios Algorithm 1 Noisy Optimization Portfolio Algorithm (NOPA). 1: Parameters: a dimension d 2 N⇤ 2: Initialization: initialize a portfolio {S1, . . . , SM} containing M solvers 3: m, n 1 4: while (true) do 5: for i = 1 to M do I Fair budget distribution 6: Apply an iteration of solver Si until it has received at least n data samples 7: xi ,n the current recommendation by solver Si 8: end for 9: if n = rm then I Periodically we compare 10: for i = 1 to M do 11: Perform sm evaluations of the (stochastic) reward R(xi ,kn ) 12: yi the average reward 13: end for 14: i⇤ arg min i2{1,...,M} yi I Who is best ? 15: m m + 1 16: end if
- 17. Algorithm Portfolios for Noisy Optimization: Compare Solvers Early Marie-Liesse CAUWET, Jialin LIU, Olivier TEYTAUD Outline Black-box Noisy Optimization Framework Algorithm Portfolios Noisy Optimization Algorithms (NOAs) Experiments Conclusions References Algorithm Portfolios Algorithm 1 Noisy Optimization Portfolio Algorithm (NOPA). 1: Parameters: a dimension d 2 N⇤ 2: Initialization: initialize a portfolio {S1, . . . , SM} containing M solvers 3: m, n 1 4: while (true) do 5: for i = 1 to M do I Fair budget distribution 6: Apply an iteration of solver Si until it has received at least n data samples 7: xi ,n the current recommendation by solver Si 8: end for 9: if n = rm then I Periodically we compare 10: for i = 1 to M do 11: Perform sm evaluations of the (stochastic) reward R(xi ,kn ) 12: yi the average reward 13: end for 14: i⇤ arg min i2{1,...,M} yi I Who is best ? 15: m m + 1 16: end if 17: ˜xn xi⇤,n I Recommendation follows i⇤ 18: n n + 1 19: end while
- 18. Algorithm Portfolios for Noisy Optimization: Compare Solvers Early Marie-Liesse CAUWET, Jialin LIU, Olivier TEYTAUD Outline Black-box Noisy Optimization Framework Algorithm Portfolios Noisy Optimization Algorithms (NOAs) Experiments Conclusions References Algorithm Portfolios Algorithm 1 Noisy Optimization Portfolio Algorithm (NOPA). 1: Parameters: a dimension d 2 N⇤ 2: Initialization: initialize a portfolio {S1, . . . , SM} containing M solvers 3: m, n 1 4: while (true) do 5: for i = 1 to M do I Fair budget distribution 6: Apply an iteration of solver Si until it has received at least n data samples 7: xi ,n the current recommendation by solver Si 8: end for 9: if n = rm then I Periodically we compare 10: for i = 1 to M do 11: Perform sm evaluations of the (stochastic) reward R(xi ,kn ) 12: yi the average reward 13: end for 14: i⇤ arg min i2{1,...,M} yi I Who is best ? 15: m m + 1 16: end if 17: ˜xn xi⇤,n I Recommendation follows i⇤ 18: n n + 1 19: end while
- 19. Algorithm Portfolios for Noisy Optimization: Compare Solvers Early Marie-Liesse CAUWET, Jialin LIU, Olivier TEYTAUD Outline Black-box Noisy Optimization Framework Algorithm Portfolios Noisy Optimization Algorithms (NOAs) Experiments Conclusions References Algorithm Portfolios Compare Solvers Early kn n: lag
- 20. Algorithm Portfolios for Noisy Optimization: Compare Solvers Early Marie-Liesse CAUWET, Jialin LIU, Olivier TEYTAUD Outline Black-box Noisy Optimization Framework Algorithm Portfolios Noisy Optimization Algorithms (NOAs) Experiments Conclusions References Algorithm Portfolios Compare Solvers Early kn n: lag 8i 2 {1, . . . ,M}, xi ,kn6= or = xi ,n
- 21. Algorithm Portfolios for Noisy Optimization: Compare Solvers Early Marie-Liesse CAUWET, Jialin LIU, Olivier TEYTAUD Outline Black-box Noisy Optimization Framework Algorithm Portfolios Noisy Optimization Algorithms (NOAs) Experiments Conclusions References Algorithm Portfolios Compare Solvers Early kn n: lag 8i 2 {1, . . . ,M}, xi ,kn6= or = xi ,n Why this lag ?
- 22. Algorithm Portfolios for Noisy Optimization: Compare Solvers Early Marie-Liesse CAUWET, Jialin LIU, Olivier TEYTAUD Outline Black-box Noisy Optimization Framework Algorithm Portfolios Noisy Optimization Algorithms (NOAs) Experiments Conclusions References Algorithm Portfolios Compare Solvers Early kn n: lag 8i 2 {1, . . . ,M}, xi ,kn6= or = xi ,n Why this lag ? comparing good points ! comparing points with similar fitness comparing points with similar fitness ! very expensive algorithms’ ranking is usually stable ! no use comparing the very last
- 23. Algorithm Portfolios for Noisy Optimization: Compare Solvers Early Marie-Liesse CAUWET, Jialin LIU, Olivier TEYTAUD Outline Black-box Noisy Optimization Framework Algorithm Portfolios Noisy Optimization Algorithms (NOAs) Experiments Conclusions References Noisy Optimization Algorithms (NOAs)
- 24. Algorithm Portfolios for Noisy Optimization: Compare Solvers Early Marie-Liesse CAUWET, Jialin LIU, Olivier TEYTAUD Outline Black-box Noisy Optimization Framework Algorithm Portfolios Noisy Optimization Algorithms (NOAs) Experiments Conclusions References Noisy Optimization Algorithms (NOAs) SA-ES: Self-Adaptive Evolution Strategy;
- 25. Algorithm Portfolios for Noisy Optimization: Compare Solvers Early Marie-Liesse CAUWET, Jialin LIU, Olivier TEYTAUD Outline Black-box Noisy Optimization Framework Algorithm Portfolios Noisy Optimization Algorithms (NOAs) Experiments Conclusions References Noisy Optimization Algorithms (NOAs) SA-ES: Self-Adaptive Evolution Strategy; Fabian’s algorithm: a first-order method using gradients estimated by finite di↵erences[3, 2];
- 26. Algorithm Portfolios for Noisy Optimization: Compare Solvers Early Marie-Liesse CAUWET, Jialin LIU, Olivier TEYTAUD Outline Black-box Noisy Optimization Framework Algorithm Portfolios Noisy Optimization Algorithms (NOAs) Experiments Conclusions References Noisy Optimization Algorithms (NOAs) SA-ES: Self-Adaptive Evolution Strategy; Fabian’s algorithm: a first-order method using gradients estimated by finite di↵erences[3, 2]; Noisy Newton’s algorithm: a second-order method using a Hessian matrix approximated also by finite di↵erences[1]; . . .
- 27. Algorithm Portfolios for Noisy Optimization: Compare Solvers Early Marie-Liesse CAUWET, Jialin LIU, Olivier TEYTAUD Outline Black-box Noisy Optimization Framework Algorithm Portfolios Noisy Optimization Algorithms (NOAs) Experiments Conclusions References NOA 1: SA-ES with revaluations Algorithm 2 Self-Adaptive Evolution Strategy with revaluations. 1: Parameters: K > 0, ⇣ # 0, # # μ 2 N⇤, a dimension d 2 N⇤ 2: Input: an initial parent x1,i 2 Rd and an initial $1,i = 1, i 2 {1, . . . ,μ} 3: n 1 4: while (true) do 5: Generate # individuals ij , j 2 {1, . . . ,#}, independently usingI Generation $j = $n,mod(j−1,μ)+1 ⇥ exp ✓ 1 2d N ◆ and ij = xn,mod(j−1,μ)+1 + $jN 6: Evaluate each of them dKn⇣e times and average their fitness values I Evaluation 7: Define j1, . . . , j" so that3 I Ranking EdKn⇣e[f (ij1 )] EdKn⇣e[f (ij2 )] · · · EdKn⇣e[f (ij")] 8: $n+1,k = $jk and xn+1,k = ijk , k 2 {1, . . . ,μ} I Updating 9: n n + 1 10: end while 3Em denotes the average over m resamplings
- 28. Algorithm Portfolios for Noisy Optimization: Compare Solvers Early Marie-Liesse CAUWET, Jialin LIU, Olivier TEYTAUD Outline Black-box Noisy Optimization Framework Algorithm Portfolios Noisy Optimization Algorithms (NOAs) Experiments Conclusions References NOA 2: Fabian’s Algorithm Algorithm 3 Fabian’s stochastic gradient algorithm with finite di↵erences[5, 2]. 1: Parameters: a dimension d 2 N⇤, 12 > % > 0, a > 0, c > 0, m 2 N⇤, weights w1 > · · · > wm summing to 1, scales 1 # u1 > · · · > um > 0 2: Input: an initial x1 2 Rd 3: n 1 4: while (true) do 5: Compute $n = c/n# 6: Evaluate the gradient g at xn by finite di↵erences, averaging over 2m sam-ples per axis. 8i 2 {1, . . . , d}, 8j{1 . . .m} x(i ,j)+ n = xn + uj ei and x(i ,j)− n = xn − uj ei gi = 1 2$n Xm j=1 wj ⇣ f (x(i ,j)+ n ) − f (x(i ,j)− n ) ⌘ 7: Gradient step: Apply xn+1 = xn − an g 8: n n + 1 9: end while
- 29. Algorithm Portfolios for Noisy Optimization: Compare Solvers Early Marie-Liesse CAUWET, Jialin LIU, Olivier TEYTAUD Outline Black-box Noisy Optimization Framework Algorithm Portfolios Noisy Optimization Algorithms (NOAs) Experiments Conclusions References NOA 3: Noisy Newton’s algorithm Algorithm 4 Noisy Newton’s algorithm with gradient and Hes-sian approximated by finite di↵erences and revaluations[1]. 1: Parameters: a dimension d 2 N⇤, A > 0, B > 0, ↵ > 0, ' > 0, ✏ > 0 2: Input: ˆh identity matrix, an initial x1 2 Rd 3: n 1 4: while (true) do 5: Compute sigman = A/n↵ 6: Evaluate the gradient g at xn by finite di↵erences, averaging over dBn%e samples at distance ⇥($n) of xn 7: for i = 1 to d do 8: Evaluate Hessian hi ,i by finite di↵erences at xn + $ei and xn − $ei , averaging each evaluation over dBn%e resamplings 9: for j = 1 to d do 10: if i == j then 11: Update ˆhi ,j using ˆhi ,i = (1 − ✏)ˆhi ,i + ✏hi ,i 12: else 13: Evaluate hi ,j by finite di↵erences thanks to evaluations at each of xn ± $ei ± $ej, averaging over dBn%/10e samples 14: Update ˆhi ,j using ˆhi ,j = (1 − ✏ d )ˆhi ,j + ✏ d hi ,j 15: end if 16: end for 17: end for
- 30. Algorithm Portfolios for Noisy Optimization: Compare Solvers Early Marie-Liesse CAUWET, Jialin LIU, Olivier TEYTAUD Outline Black-box Noisy Optimization Framework Algorithm Portfolios Noisy Optimization Algorithms (NOAs) Experiments Conclusions References NOA 3: Noisy Newton’s algorithm Algorithm 4 Noisy Newton’s algorithm with gradient and Hes-sian approximated by finite di↵erences and revaluations[1]. 18: ) solution of ˆh) = −g I Newton step 19: if ) > C$n then 20: ) = C$n ' ||'|| 21: end if 22: Apply xn+1 = xn + ) 23: n n + 1 24: end while
- 31. Algorithm Portfolios for Noisy Optimization: Compare Solvers Early Marie-Liesse CAUWET, Jialin LIU, Olivier TEYTAUD Outline Black-box Noisy Optimization Framework Algorithm Portfolios Noisy Optimization Algorithms (NOAs) Experiments Conclusions References Experiments A trivial problem f (x) = ||x||2 + ||x||zN, x 2 Rd . d: dimension; N: a Gaussian standard noise; z 2 {0, 1, 2}.
- 32. Algorithm Portfolios for Noisy Optimization: Compare Solvers Early Marie-Liesse CAUWET, Jialin LIU, Olivier TEYTAUD Outline Black-box Noisy Optimization Framework Algorithm Portfolios Noisy Optimization Algorithms (NOAs) Experiments Conclusions References Noisy optimization solvers Table : Mono-solvers and portfolios used in the experiments. Solvers Algorithm and parametrization Fabian1 Fabian’s solver with stepsize $n = 10/n0.1, a = 10. Fabian2 Fabian’s solver with stepsize $n = 10/n0.49, a = 100. Newton Newton’s solver with stepsize $n = 100/n4, resamplingn = n2. RSAES RSAES with # = 10d, μ = 5d, resamplingn = 10n2.
- 33. Algorithm Portfolios for Noisy Optimization: Compare Solvers Early Marie-Liesse CAUWET, Jialin LIU, Olivier TEYTAUD Outline Black-box Noisy Optimization Framework Algorithm Portfolios Noisy Optimization Algorithms (NOAs) Experiments Conclusions References Noisy optimization solvers Table : Mono-solvers and portfolios used in the experiments. Solvers Algorithm and parametrization Fabian1 Fabian’s solver with stepsize $n = 10/n0.1, a = 10. Fabian2 Fabian’s solver with stepsize $n = 10/n0.49, a = 100. Newton Newton’s solver with stepsize $n = 100/n4, resamplingn = n2. RSAES RSAES with # = 10d, μ = 5d, resamplingn = 10n2. Portfolio NOPA of 4 mono-solvers with kn = dn0.1e, rn = n3, sn = 15n2. P. + Sharing Portfolio with information sharing enabled. Recall n: portfolio iteration number; rn: revaluation number for comparing at iteration n; sn: comparison period; kn: index of recommendation to be compared.
- 34. Algorithm Portfolios for Noisy Optimization: Compare Solvers Early Marie-Liesse CAUWET, Jialin LIU, Olivier TEYTAUD Outline Black-box Noisy Optimization Framework Algorithm Portfolios Noisy Optimization Algorithms (NOAs) Experiments Conclusions References Simple Regret4[6] For Simple Regret= SR Let x⇤ be the optimum of f . Let xn be the individual evaluated at nth evaluation and ˜xn the optimum estimated after nth evaluation Simple Regret SR = E(f (˜xn) − f (x⇤)) Slope(SR) = lim n!1 log(SR(n)) log(n) 4Di↵erence between average payo↵ recommended and optimal
- 35. Algorithm Portfolios for Noisy Optimization: Compare Solvers Early Marie-Liesse CAUWET, Jialin LIU, Olivier TEYTAUD Outline Black-box Noisy Optimization Framework Algorithm Portfolios Noisy Optimization Algorithms (NOAs) Experiments Conclusions References f (x) = ||x||2 + ||x||zN in dimension 2 Solvers z = 0 z = 1 z = 2 Fabian1 -1.24±0.05 -1.25±0.06 -1.23±0.06 Fabian2 -0.17±0.09 -1.75±0.10 -3.16±0.06 Newton -0.20±0.09 -1.84±0.34 -1.93±0.00 RSAES -0.41±0.08 -0.61±0.13 -0.60±0.16 Portfolio -1.00±0.28 -1.63±0.06 -2.69±0.07 P. + Sharing -0.93±0.31 -1.64±0.05 -2.71±0.07 Table : Slope(SR) of experiments in dimension 2. Best mono-solver Worst mono-solver Portfolios
- 36. Algorithm Portfolios for Noisy Optimization: Compare Solvers Early Marie-Liesse CAUWET, Jialin LIU, Olivier TEYTAUD Outline Black-box Noisy Optimization Framework Algorithm Portfolios Noisy Optimization Algorithms (NOAs) Experiments Conclusions References f (x) = ||x||2 + ||x||zN in dimension 15 Solvers z = 0 z = 1 z = 2 Fabian1 -0.83±0.02 -1.03±0.02 -1.02±0.02 Fabian2 0.11±0.02 -1.30±0.02 -2.39±0.02 Newton 0.00±0.02 -1.27±0.23 -1.33±0.00 RSAES 0.15±0.01 0.14±0.02 0.15±0.01 Portfolio -0.72±0.02 -1.06±0.01 -1.90±0.02 P. + Sharing -0.72±0.02 -1.05±0.03 -1.90±0.03 Table : Slope(SR) of experiments in dimension 15. Best mono-solver Worst mono-solver Portfolios
- 37. Algorithm Portfolios for Noisy Optimization: Compare Solvers Early Marie-Liesse CAUWET, Jialin LIU, Olivier TEYTAUD Outline Black-box Noisy Optimization Framework Algorithm Portfolios Noisy Optimization Algorithms (NOAs) Experiments Conclusions References Experiments Results
- 38. Algorithm Portfolios for Noisy Optimization: Compare Solvers Early Marie-Liesse CAUWET, Jialin LIU, Olivier TEYTAUD Outline Black-box Noisy Optimization Framework Algorithm Portfolios Noisy Optimization Algorithms (NOAs) Experiments Conclusions References Experiments Results The portfolio algorithm successfully reaches almost the same slope(SR) as the best of its solvers;
- 39. Algorithm Portfolios for Noisy Optimization: Compare Solvers Early Marie-Liesse CAUWET, Jialin LIU, Olivier TEYTAUD Outline Black-box Noisy Optimization Framework Algorithm Portfolios Noisy Optimization Algorithms (NOAs) Experiments Conclusions References Experiments Results The portfolio algorithm successfully reaches almost the same slope(SR) as the best of its solvers; for z = 2 the best algorithm is the second variant of Fabian’s algorithm; for z = 1 the approximation of Noisy Newton’s algorithm performs best; for z = 0 the first variant of Fabian’s algorithm performs best;
- 40. Algorithm Portfolios for Noisy Optimization: Compare Solvers Early Marie-Liesse CAUWET, Jialin LIU, Olivier TEYTAUD Outline Black-box Noisy Optimization Framework Algorithm Portfolios Noisy Optimization Algorithms (NOAs) Experiments Conclusions References Experiments Results The portfolio algorithm successfully reaches almost the same slope(SR) as the best of its solvers; for z = 2 the best algorithm is the second variant of Fabian’s algorithm; for z = 1 the approximation of Noisy Newton’s algorithm performs best; for z = 0 the first variant of Fabian’s algorithm performs best; the sharing has little or no impact.
- 41. Algorithm Portfolios for Noisy Optimization: Compare Solvers Early Marie-Liesse CAUWET, Jialin LIU, Olivier TEYTAUD Outline Black-box Noisy Optimization Framework Algorithm Portfolios Noisy Optimization Algorithms (NOAs) Experiments Conclusions References Conclusions Conclusions
- 42. Algorithm Portfolios for Noisy Optimization: Compare Solvers Early Marie-Liesse CAUWET, Jialin LIU, Olivier TEYTAUD Outline Black-box Noisy Optimization Framework Algorithm Portfolios Noisy Optimization Algorithms (NOAs) Experiments Conclusions References Conclusions Conclusions Main conclusion: portfolios are classical in combinatorial optimization; (because in C.O. di↵erences between runtimes can be huge);
- 43. Algorithm Portfolios for Noisy Optimization: Compare Solvers Early Marie-Liesse CAUWET, Jialin LIU, Olivier TEYTAUD Outline Black-box Noisy Optimization Framework Algorithm Portfolios Noisy Optimization Algorithms (NOAs) Experiments Conclusions References Conclusions Conclusions Main conclusion: portfolios are classical in combinatorial optimization; (because in C.O. di↵erences between runtimes can be huge); portfolios also make a big di↵erence in noisy optimization; (because in N.O., with lag, comparison cost = small).
- 44. Algorithm Portfolios for Noisy Optimization: Compare Solvers Early Marie-Liesse CAUWET, Jialin LIU, Olivier TEYTAUD Outline Black-box Noisy Optimization Framework Algorithm Portfolios Noisy Optimization Algorithms (NOAs) Experiments Conclusions References Conclusions Conclusions Main conclusion: portfolios are classical in combinatorial optimization; (because in C.O. di↵erences between runtimes can be huge); portfolios also make a big di↵erence in noisy optimization; (because in N.O., with lag, comparison cost = small). Sharing not that good.
- 45. Algorithm Portfolios for Noisy Optimization: Compare Solvers Early Marie-Liesse CAUWET, Jialin LIU, Olivier TEYTAUD Outline Black-box Noisy Optimization Framework Algorithm Portfolios Noisy Optimization Algorithms (NOAs) Experiments Conclusions References Conclusions Conclusions Main conclusion: portfolios are classical in combinatorial optimization; (because in C.O. di↵erences between runtimes can be huge); portfolios also make a big di↵erence in noisy optimization; (because in N.O., with lag, comparison cost = small). Sharing not that good. We show mathematicallya and empirically a log(M) shift when using M solvers, when working on a classical log-log scale (classical in noisy optimization).
- 46. Algorithm Portfolios for Noisy Optimization: Compare Solvers Early Marie-Liesse CAUWET, Jialin LIU, Olivier TEYTAUD Outline Black-box Noisy Optimization Framework Algorithm Portfolios Noisy Optimization Algorithms (NOAs) Experiments Conclusions References Conclusions Conclusions Main conclusion: portfolios are classical in combinatorial optimization; (because in C.O. di↵erences between runtimes can be huge); portfolios also make a big di↵erence in noisy optimization; (because in N.O., with lag, comparison cost = small). Sharing not that good. We show mathematicallya and empirically a log(M) shift when using M solvers, when working on a classical log-log scale (classical in noisy optimization). A portfolio of solvers = approximately as efficient as the bestb. asee paper :-) bMore practical work can be found in [4].
- 47. Algorithm Portfolios for Noisy Optimization: Compare Solvers Early Marie-Liesse CAUWET, Jialin LIU, Olivier TEYTAUD Outline Black-box Noisy Optimization Framework Algorithm Portfolios Noisy Optimization Algorithms (NOAs) Experiments Conclusions References Perspectives Information sharing & unfair budget distribution With 4 solvers, the log(M) shift is ok; with 40 maybe not.
- 48. Algorithm Portfolios for Noisy Optimization: Compare Solvers Early Marie-Liesse CAUWET, Jialin LIU, Olivier TEYTAUD Outline Black-box Noisy Optimization Framework Algorithm Portfolios Noisy Optimization Algorithms (NOAs) Experiments Conclusions References Perspectives Information sharing & unfair budget distribution With 4 solvers, the log(M) shift is ok; with 40 maybe not. Identifying relevant information for sharing.
- 49. Algorithm Portfolios for Noisy Optimization: Compare Solvers Early Marie-Liesse CAUWET, Jialin LIU, Olivier TEYTAUD Outline Black-box Noisy Optimization Framework Algorithm Portfolios Noisy Optimization Algorithms (NOAs) Experiments Conclusions References Perspectives Information sharing & unfair budget distribution With 4 solvers, the log(M) shift is ok; with 40 maybe not. Identifying relevant information for sharing. If solver 1 says “I’ll never do better than X” and solver 2 says “I have found at least Y > X” then we can stop 1.
- 50. Algorithm Portfolios for Noisy Optimization: Compare Solvers Early Marie-Liesse CAUWET, Jialin LIU, Olivier TEYTAUD Outline Black-box Noisy Optimization Framework Algorithm Portfolios Noisy Optimization Algorithms (NOAs) Experiments Conclusions References Some references Sandra Astete-Morales, Marie-Liesse Cauwet, Jialin Liu, and Olivier Teytaud. Noisy optimization rates. submitted, 2013. Vaclav Fabian. Stochastic Approximation of Minima with Improved Asymptotic Speed. Annals of Mathematical statistics, 38:191–200, 1967. Jack Kiefer and Jacob Wolfowitz. Stochastic Estimation of the Maximum of a Regression Function. Annals of Mathematical statistics, 23:462–466, 1952. Jialin Liu and Olivier Teytaud. Meta online learning: experiments on a unit commitment problem. In ESANN, Bruges, Belgium, 2014. Ohad Shamir. On the complexity of bandit and derivative-free stochastic convex optimization. CoRR, abs/1209.2388, 2012. Gilles Stoltz, S´ebastien Bubeck, and R´emi Munos. Pure exploration in finitely-armed and continuous-armed bandits. Theoretical Computer Science, 412(19):1832–1852, April 2011.
- 51. Algorithm Portfolios for Noisy Optimization: Compare Solvers Early Marie-Liesse CAUWET, Jialin LIU, Olivier TEYTAUD Outline Black-box Noisy Optimization Framework Algorithm Portfolios Noisy Optimization Algorithms (NOAs) Experiments Conclusions References Thank you for your attention ! MLC TAO Team https://tao.lri.fr/tiki-index.php INRIA Saclay-LRI-CNRS, Univ. Paris-Sud DIGITEO, 91190 Gif-sur-Yvette, France OT
- 52. Algorithm Portfolios for Noisy Optimization: Compare Solvers Early Marie-Liesse CAUWET, Jialin LIU, Olivier TEYTAUD Outline Black-box Noisy Optimization Framework Algorithm Portfolios Noisy Optimization Algorithms (NOAs) Experiments Conclusions References Contacts Algorithm Portfolios for Noisy Optimization: Compare Solvers Early TAO Team, INRIA Saclay-LRI-CNRS, Univ. Paris-Sud 91190 Gif-sur-Yvette, France Marie-Liesse CAUWET, Jialin LIU, Olivier TEYTAUD Contacts: lastname.firstname@inria.fr Personal page: https://www.lri.fr/⇠lastname/ Slides of presentation: https://www.lri.fr/⇠liu/portfolio2 lion8.pdf
- 53. Algorithm Portfolios for Noisy Optimization: Compare Solvers Early Marie-Liesse CAUWET, Jialin LIU, Olivier TEYTAUD Outline Black-box Noisy Optimization Framework Algorithm Portfolios Noisy Optimization Algorithms (NOAs) Experiments Conclusions References Rates Regret For Regret = SR or CR Slope(Regret) = lim n!1 log(Regret(n)) log(n) Algorithm Parameter Slope(SR) Slope(CR) " ! 0 −1 1 Fabian " ! 12 0 1 " ! 14 −12 12