Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Csr2011 june14 11_30_winzen_rotated

307 views

Published on

  • Be the first to comment

  • Be the first to like this

Csr2011 june14 11_30_winzen_rotated

  1. 1. Towards a Complexity Theory for Randomized Search Heuristics:The Ranking-Based Black-Box Model Benjamin Doerr / Carola Winzen CSR, June 14, 2011
  2. 2. Our Motivation General-purpose (randomized) search heuristics are ...easy to implement, ...very flexible, ...need little a priori knowledge about problem at hand, ...can deal with many constraints and nonlinearities, B. Doerr/C. Winzen: Ranking-Based Black-Box Model CSR, June 14, 2011
  3. 3. Our Motivation General-purpose (randomized) search heuristics are ...easy to implement, ...very flexible, ...need little a priori knowledge about problem at hand, ...can deal with many constraints and nonlinearities, and thus, frequently applied in practice. Some theoretical results exist. Typical result: runtime analysis for a particular algorithm on a particular problem. B. Doerr/C. Winzen: Ranking-Based Black-Box Model CSR, June 14, 2011
  4. 4. Our Motivation General-purpose (randomized) search heuristics are ...easy to implement, ...very flexible, ...need little a priori knowledge about problem at hand, ...can deal with many constraints and nonlinearities, and thus, frequently applied in practice. Some theoretical results exist. Typical result: runtime analysis for Comparable to a particular algorithm the “early days” on a particular problem. of Computer Science B. Doerr/C. Winzen: Ranking-Based Black-Box Model CSR, June 14, 2011
  5. 5. Our Motivation In classical theoretical computer science: first results: runtime analysis for a particular algorithm on a particular problem. B. Doerr/C. Winzen: Ranking-Based Black-Box Model CSR, June 14, 2011
  6. 6. Our Motivation In classical theoretical computer science: first results: runtime analysis for a particular algorithm on a particular problem. general lower bounds (“tractability of a problem”) complexity theory B. Doerr/C. Winzen: Ranking-Based Black-Box Model CSR, June 14, 2011
  7. 7. Our Motivation In classical theoretical computer science: first results: runtime analysis for a particular algorithm on a particular problem. general lower bounds (“tractability of a problem”) complexity theory How to create a complexity theory for randomized search heuristics? B. Doerr/C. Winzen: Ranking-Based Black-Box Model CSR, June 14, 2011
  8. 8. Our Motivation In classical theoretical computer science: first results: runtime analysis for a particular algorithm on a particular problem. general lower bounds (“tractability of a problem”) complexity theory Our aim: to understand the tractability of a problem for general-purpose (randomized) search heuristics “Towards a Complexity Theory for Randomized Search Heuristics” B. Doerr/C. Winzen: Ranking-Based Black-Box Model CSR, June 14, 2011
  9. 9. A General View on Search Heuristics Search Heuristic f Black-Box = “Oracle” B. Doerr/C. Winzen: Ranking-Based Black-Box Model CSR, June 14, 2011
  10. 10. A General View on Search Heuristics x Search Heuristic f Black-Box = “Oracle” B. Doerr/C. Winzen: Ranking-Based Black-Box Model CSR, June 14, 2011
  11. 11. A General View on Search Heuristics x Search Heuristic f f(x) x Black-Box = “Oracle” B. Doerr/C. Winzen: Ranking-Based Black-Box Model CSR, June 14, 2011
  12. 12. A General View on Search Heuristics x Search Heuristic f f(x) x Black-Box = “Oracle” Typical cost measure: number of function evaluations until an optimal solution is queried for the first time B. Doerr/C. Winzen: Ranking-Based Black-Box Model CSR, June 14, 2011
  13. 13. A General View on Search Heuristics Classical Query Complexity Model x Search Heuristic f f(x) x Black-Box = “Oracle” Typical cost measure: number of function evaluations until an optimal solution is queried for the first time B. Doerr/C. Winzen: Ranking-Based Black-Box Model CSR, June 14, 2011
  14. 14. A Theory for (Randomized) Search Heuristics Part 1: Classical query complexity model Game theoretic view Example: Mastermind Part 2: Refinement: ranking-based query complexity “Towards a Complexity Theory for Randomized Search Heuristics: The Ranking-Based Black-Box Model” B. Doerr/C. Winzen: Ranking-Based Black-Box Model CSR, June 14, 2011
  15. 15. Example: A Mastermind Problem Carole (=oracle) chooses a binary string of length n: 1 1 0 1 0 0 1 1 B. Doerr/C. Winzen: Ranking-Based Black-Box Model CSR, June 14, 2011
  16. 16. Example: A Mastermind Problem Carole (=oracle) chooses a binary string of length n: 1 1 0 1 0 0 1 1 Paul (=algo.) tries to find it. He may ask any string of length n: B. Doerr/C. Winzen: Ranking-Based Black-Box Model CSR, June 14, 2011
  17. 17. Example: A Mastermind Problem Carole (=oracle) chooses a binary string of length n: 1 1 0 1 0 0 1 1 Paul (=algo.) tries to find it. He may ask any string of length n: 1 0 1 0 1 0 1 0 B. Doerr/C. Winzen: Ranking-Based Black-Box Model CSR, June 14, 2011
  18. 18. Example: A Mastermind Problem Carole (=oracle) chooses a binary string of length n: 1 1 0 1 0 0 1 1 Paul (=algo.) tries to find it. He may ask any string of length n: 1 0 1 0 1 0 1 0 Carole computes in how many positions the strings coincide: B. Doerr/C. Winzen: Ranking-Based Black-Box Model CSR, June 14, 2011
  19. 19. Example: A Mastermind Problem Carole (=oracle) chooses a binary string of length n: 1 1 0 1 0 0 1 1 Paul (=algo.) tries to find it. He may ask any string of length n: 1 0 1 0 1 0 1 0 Carole computes in how many positions the strings coincide: 1 0 1 0 1 0 1 0 B. Doerr/C. Winzen: Ranking-Based Black-Box Model CSR, June 14, 2011
  20. 20. Example: A Mastermind Problem Carole (=oracle) chooses a binary string of length n: 1 1 0 1 0 0 1 1 Paul (=algo.) tries to find it. He may ask any string of length n: 1 0 1 0 1 0 1 0 Carole computes in how many positions the strings coincide: 1 0 1 0 1 0 1 0 “Paul, our strings coincide in 3 bits” B. Doerr/C. Winzen: Ranking-Based Black-Box Model CSR, June 14, 2011
  21. 21. Example: A Mastermind Problem Carole (=oracle) chooses a binary string of length n: 1 1 0 1 0 0 1 1 Paul (=algo.) tries to find it. He may ask any string of length n: 1 0 1 0 1 0 1 0 Carole computes in how many positions the strings coincide: 1 0 1 0 1 0 1 0 We say that the “Paul, our strings coincide in 3 bits” “fitness” of Paul’s string is 3 B. Doerr/C. Winzen: Ranking-Based Black-Box Model CSR, June 14, 2011
  22. 22. Example: A Mastermind Problem Carole chooses a binary string of length n: 1 1 0 1 0 0 1 1 Paul tries to find it. He may ask any string of length n: 1 0 1 0 1 0 1 0 3 B. Doerr/C. Winzen: Ranking-Based Black-Box Model CSR, June 14, 2011
  23. 23. Example: A Mastermind Problem Carole chooses a binary string of length n: 1 1 0 1 0 0 1 1 Paul tries to find it. He may ask any string of length n: 1 0 1 0 1 0 1 0 3 1 0 1 1 0 1 0 1 B. Doerr/C. Winzen: Ranking-Based Black-Box Model CSR, June 14, 2011
  24. 24. Example: A Mastermind Problem Carole chooses a binary string of length n: 1 1 0 1 0 0 1 1 Paul tries to find it. He may ask any string of length n: 1 0 1 0 1 0 1 0 3 1 0 1 1 0 1 0 1 B. Doerr/C. Winzen: Ranking-Based Black-Box Model CSR, June 14, 2011
  25. 25. Example: A Mastermind Problem Carole chooses a binary string of length n: 1 1 0 1 0 0 1 1 Paul tries to find it. He may ask any string of length n: 1 0 1 0 1 0 1 0 3 1 0 1 1 0 1 0 1 4 ... B. Doerr/C. Winzen: Ranking-Based Black-Box Model CSR, June 14, 2011
  26. 26. Example: A Mastermind Problem Carole chooses a binary string of length n: 1 1 0 1 0 0 1 1 Paul tries to find it. He may ask any string of length n: 1 0 1 0 1 0 1 0 3 1 0 1 1 0 1 0 1 4 ... How many queries does Paul need, on average, until he has identified Carole’s string? B. Doerr/C. Winzen: Ranking-Based Black-Box Model CSR, June 14, 2011
  27. 27. Reminder Our aim: To understand tractability of a problem for general-purpose (randomized) search heuristics Measure: number of function evaluations until an optimal solution is queried for the first time Our main interest: good lower bounds B. Doerr/C. Winzen: Ranking-Based Black-Box Model CSR, June 14, 2011
  28. 28. The Master Mind Problem: What Search Heuristics Do Paul tries to find Carole’s binary string of length n: 1 1 0 1 0 0 1 1 B. Doerr/C. Winzen: Ranking-Based Black-Box Model CSR, June 14, 2011
  29. 29. The Master Mind Problem: What Search Heuristics Do Paul tries to find Carole’s binary string of length n: 1 1 0 1 0 0 1 1 First query is arbitrary: 1 0 1 0 1 0 1 0 B. Doerr/C. Winzen: Ranking-Based Black-Box Model CSR, June 14, 2011
  30. 30. The Master Mind Problem: What Search Heuristics Do Paul tries to find Carole’s binary string of length n: 1 1 0 1 0 0 1 1 First query is arbitrary: 1 0 1 0 1 0 1 0 3 B. Doerr/C. Winzen: Ranking-Based Black-Box Model CSR, June 14, 2011
  31. 31. The Master Mind Problem: What Search Heuristics Do Paul tries to find Carole’s binary string of length n: 1 1 0 1 0 0 1 1 First query is arbitrary: 1 0 1 0 1 0 1 0 3 Then flip exactly one bit (chosen u.a.r.): 1 1 1 0 1 0 1 0 B. Doerr/C. Winzen: Ranking-Based Black-Box Model CSR, June 14, 2011
  32. 32. The Master Mind Problem: What Search Heuristics Do Paul tries to find Carole’s binary string of length n: 1 1 0 1 0 0 1 1 First query is arbitrary: 1 0 1 0 1 0 1 0 3 Then flip exactly one bit (chosen u.a.r.): 1 1 1 0 1 0 1 0 4 B. Doerr/C. Winzen: Ranking-Based Black-Box Model CSR, June 14, 2011
  33. 33. The Master Mind Problem: What Search Heuristics Do Paul tries to find Carole’s binary string of length n: 1 1 0 1 0 0 1 1 First query is arbitrary: 1 0 1 0 1 0 1 0 3 Then flip exactly one bit (chosen u.a.r.): 1 1 1 0 1 0 1 0 4 And it continues with the better of the two: 1 1 1 0 1 1 1 0 B. Doerr/C. Winzen: Ranking-Based Black-Box Model CSR, June 14, 2011
  34. 34. The Master Mind Problem: What Search Heuristics Do Paul tries to find Carole’s binary string of length n: 1 1 0 1 0 0 1 1 First query is arbitrary: 1 0 1 0 1 0 1 0 3 Then flip exactly one bit (chosen u.a.r.): 1 1 1 0 1 0 1 0 4 Random Local Search algorithm: Θ(n log n) Coupon Collector [Folklore] B. Doerr/C. Winzen: Ranking-Based Black-Box Model CSR, June 14, 2011
  35. 35. The Master Mind Problem: What Search Heuristics Do Paul tries to find Carole’s binary string of length n: 1 1 0 1 0 0 1 1 First query is arbitrary: 1 0 1 0 1 0 1 0 B. Doerr/C. Winzen: Ranking-Based Black-Box Model CSR, June 14, 2011
  36. 36. The Master Mind Problem: What Search Heuristics Do Paul tries to find Carole’s binary string of length n: 1 1 0 1 0 0 1 1 First query is arbitrary: 1 0 1 0 1 0 1 0 3 Flip each bit with probability 1/n: B. Doerr/C. Winzen: Ranking-Based Black-Box Model CSR, June 14, 2011
  37. 37. The Master Mind Problem: What Search Heuristics Do Paul tries to find Carole’s binary string of length n: 1 1 0 1 0 0 1 1 First query is arbitrary: 1 0 1 0 1 0 1 0 3 Flip each bit with probability 1/n: 0 0 1 0 1 1 1 0 1 B. Doerr/C. Winzen: Ranking-Based Black-Box Model CSR, June 14, 2011
  38. 38. The Master Mind Problem: What Search Heuristics Do Paul tries to find Carole’s binary string of length n: 1 1 0 1 0 0 1 1 First query is arbitrary: 1 0 1 0 1 0 1 0 3 Flip each bit with probability 1/n: 0 0 1 0 1 1 1 0 1 And it continues with the better of the two B. Doerr/C. Winzen: Ranking-Based Black-Box Model CSR, June 14, 2011
  39. 39. The Master Mind Problem: What Search Heuristics Do Paul tries to find Carole’s binary string of length n: 1 1 0 1 0 0 1 1 First query is arbitrary: 1 0 1 0 1 0 1 0 3 Flip each bit with probability 1/n: 0 0 1 0 1 1 1 0 1 (1+1) Evolutionary Algorithm: Θ(n log n)[Droste/Jansen/Wegener 92] [Mühlenbein 02] B. Doerr/C. Winzen: Ranking-Based Black-Box Model CSR, June 14, 2011
  40. 40. The Master Mind Problem: Optimal Strategies (1/2) Paul tries to find Carole’s binary string of length n: 1 0 0 1 0 0 1 1 B. Doerr/C. Winzen: Ranking-Based Black-Box Model CSR, June 14, 2011
  41. 41. The Master Mind Problem: Optimal Strategies (1/2) Paul tries to find Carole’s binary string of length n: 1 0 0 1 0 0 1 1 He can go through the string bit by bit: 0 0 0 0 0 0 0 0 4 B. Doerr/C. Winzen: Ranking-Based Black-Box Model CSR, June 14, 2011
  42. 42. The Master Mind Problem: Optimal Strategies (1/2) Paul tries to find Carole’s binary string of length n: 1 0 0 1 0 0 1 1 He can go through the string bit by bit: 0 0 0 0 0 0 0 0 4 1 0 0 0 0 0 0 0 5 B. Doerr/C. Winzen: Ranking-Based Black-Box Model CSR, June 14, 2011
  43. 43. The Master Mind Problem: Optimal Strategies (1/2) Paul tries to find Carole’s binary string of length n: 1 0 0 1 0 0 1 1 He can go through the string bit by bit: 0 0 0 0 0 0 0 0 4 1 0 0 0 0 0 0 0 5 1 1 0 0 0 0 0 0 4 B. Doerr/C. Winzen: Ranking-Based Black-Box Model CSR, June 14, 2011
  44. 44. The Master Mind Problem: Optimal Strategies (1/2) Paul tries to find Carole’s binary string of length n: 1 0 0 1 0 0 1 1 He can go through the string bit by bit: 0 0 0 0 0 0 0 0 4 1 0 0 0 0 0 0 0 5 1 1 0 0 0 0 0 0 4 ... 1 0 0 1 0 0 1 1 8 B. Doerr/C. Winzen: Ranking-Based Black-Box Model CSR, June 14, 2011
  45. 45. The Master Mind Problem: Optimal Strategies (1/2) Paul tries to find Carole’s binary string of length n: 1 0 0 1 0 0 1 1 He can go through the string bit by bit: 0 0 0 0 0 0 0 0 4 1 0 0 0 0 0 0 0 5 1 1 0 0 0 0 0 0 4 O(n) ... 1 0 0 1 0 0 1 1 8 B. Doerr/C. Winzen: Ranking-Based Black-Box Model CSR, June 14, 2011
  46. 46. The Master Mind Problem: Optimal Strategies (1/2) Paul tries to find Carole’s binary string of length n: 1 0 0 1 0 0 1 1 He can go through the string bit by bit: 0 0 0 0 0 0 0 0 4 1 0 0 0 0 0 0 0 5 1 1 0 0 0 0 0 0 4 O(n) ... Can we do 1 0 0 1 0 0 1 1 even8 better? B. Doerr/C. Winzen: Ranking-Based Black-Box Model CSR, June 14, 2011
  47. 47. The Master Mind Problem: Optimal Strategies (2/2) 1 1 0 1 0 0 1 1 B. Doerr/C. Winzen: Ranking-Based Black-Box Model CSR, June 14, 2011
  48. 48. The Master Mind Problem: Optimal Strategies (2/2) 1 1 0 1 0 0 1 1 1 1 1 1 0 1 0 0 1 4 2 0 1 0 1 0 0 0 0 5 cn 0 0 0 1 1 1 1 1 4log n B. Doerr/C. Winzen: Ranking-Based Black-Box Model CSR, June 14, 2011
  49. 49. Optimal Strategies (2/2) 0 1 1 1 0 0 1 1 0 0 0 1 0 0 1 0 1 1 0 1 0 0 1 1 0 0 1 1 0 0 0 1 0 0 1 1 1 1 1 1 0 1 0 0 1 4 0 0 1 1 2 0 1 0 1 0 0 0 0 5 1 1 ... 1 3 4 . . 4 . 5 0 1 ... 0 6 5 . . 5 . 2 cn 0 0 ... 1 3 4 . . 4 . 5 0 0 0 1 1 1 1 1 4log n B. Doerr/C. Winzen: Ranking-Based Black-Box Model CSR, June 14, 2011
  50. 50. Optimal Strategies (2/2) 0 1 1 1 0 0 1 1 0 0 0 1 0 0 1 0 1 1 0 1 0 0 1 1 0 0 1 1 0 0 0 1 0 0 1 1 1 1 1 1 0 1 0 0 1 4 0 0 1 1 2 0 1 0 1 0 0 0 0 5 1 1 ... 1 3 4 . . 4 . 5 0 1 ... 0 6 5 . . 5 . 2 cn 0 0 ... 1 3 4 . . 4 . 5 0 0 0 1 1 1 1 1 4log n B. Doerr/C. Winzen: Ranking-Based Black-Box Model CSR, June 14, 2011
  51. 51. Optimal Strategies (2/2) 0 1 1 1 0 0 1 1 0 0 0 1 0 0 1 0 1 1 0 1 0 0 1 1 0 0 1 1 0 0 0 1 0 0 1 1 1 1 1 1 0 1 0 0 1 4 0 0 1 1 2 0 1 0 1 0 0 0 0 5 1 1 ... 1 3 4 . . 4 . 5 0 1 ... 0 6 5 . . 5 . 2 cn 0 0 ... 1 3 4 . . 4 . 5 0 0 0 1 1 1 1 1 4log n B. Doerr/C. Winzen: Ranking-Based Black-Box Model CSR, June 14, 2011
  52. 52. Optimal Strategies (2/2) 0 1 1 1 0 0 1 1 0 0 0 1 0 0 1 0 1 1 0 1 0 0 1 1 0 0 1 1 0 0 0 1 0 0 1 1 1 1 1 1 0 1 0 0 1 4 0 0 1 1 2 0 1 0 1 0 0 0 0 5 1 1 ... 1 3 4 . . 4 . 5 0 1 ... 0 6 5 . . 5 . 2 cn 0 0 ... 1 3 4 . . 4 . 5 0 0 0 1 1 1 1 1 4log n B. Doerr/C. Winzen: Ranking-Based Black-Box Model CSR, June 14, 2011
  53. 53. Optimal Strategies (2/2) 0 1 1 1 0 0 1 1 0 0 0 1 0 0 1 0 1 1 0 1 0 0 1 1 0 0 1 1 0 0 0 1 0 0 1 1 1 1 1 1 0 1 0 0 1 4 0 0 1 1 2 0 1 0 1 0 0 0 0 5 1 1 ... 1 3 4 . . 4 . 5 0 1 ... 0 6 5 . . 5 . 2 Fitness elimination technique n O( log n ) cn 0 0 ... 1 3 4 . . 4 . 5 0 0 0 1 1 1 1 1 4log n [Anil/Wiegand 09], see also [D./Johannsen/Kötzing/Lehre/Wagner/W. 11] B. Doerr/C. Winzen: Ranking-Based Black-Box Model CSR, June 14, 2011
  54. 54. Optimal Strategies (2/2) 0 1 1 1 0 0 1 1 0 0 0 1 0 0 1 0 1 1 0 1 0 0 1 1 0 0 1 1 0 0 0 1 0 0 1 1 1 1 1 1 0 1 0 0 1 4 0 0 1 1 2 0 1 0 1 0 0 0 0 5 1 1 ... 1 3 4 . . 4 . 5 0 1 ... 0 6 5 . . 5 . 2 Fitness elimination technique n Θ( log n ) cn 0 0 ... 1 3 4 . . 4 . 5 0 0 0 1 1 1 1 1 4log n [Anil/Wiegand 09], see also [D./Johannsen/Kötzing/Lehre/Wagner/W. 11] B. Doerr/C. Winzen: Ranking-Based Black-Box Model CSR, June 14, 2011
  55. 55. Intermediate Summary Want to understand tractability of a problem for general- purpose (randomized) search heuristics Query complexity as such is not a sufficient measure: Mastermind problem 1 1 0 1 0 0 1 1 Search Heuristics Query Complexity n Θ(n log n) Θ( log n ) B. Doerr/C. Winzen: Ranking-Based Black-Box Model CSR, June 14, 2011
  56. 56. The Ranking-Based Black-Box Model Observation: many randomized search heuristics use fitness values only to compare B. Doerr/C. Winzen: Ranking-Based Black-Box Model CSR, June 14, 2011
  57. 57. The Ranking-Based Black-Box Model Observation: many randomized search heuristics use fitness values only to compare RLS (1+1) EA ... B. Doerr/C. Winzen: Ranking-Based Black-Box Model CSR, June 14, 2011
  58. 58. The Ranking-Based Black-Box Model Observation: many randomized search heuristics use fitness values only to compare RLS (1+1) EA ... B. Doerr/C. Winzen: Ranking-Based Black-Box Model CSR, June 14, 2011
  59. 59. The Ranking-Based Black-Box ModelDoes not reveal absolute fitness values: Algorithm f Black-Box = “Oracle” B. Doerr/C. Winzen: Ranking-Based Black-Box Model CSR, June 14, 2011
  60. 60. The Ranking-Based Black-Box ModelDoes not reveal absolute fitness values: x Algorithm f Black-Box = “Oracle” B. Doerr/C. Winzen: Ranking-Based Black-Box Model CSR, June 14, 2011
  61. 61. The Ranking-Based Black-Box ModelDoes not reveal absolute fitness values: x Algorithm f Rank 1 x Rank 2 x Black-Box Rank 2 x = “Oracle” Rank 4 x B. Doerr/C. Winzen: Ranking-Based Black-Box Model CSR, June 14, 2011
  62. 62. The Ranking-Based Black-Box ModelEquivalent formulation: Let g : R → R be a strictly monotone function x Algorithm f g g(f(x))=127 Rank 1 g(f(x)) fx g(f(x))=125 Rank 2 Black-Box g(f(x))=125 Rank 2 g(f(x))=27 Rank 4 = “Oracle” B. Doerr/C. Winzen: Ranking-Based Black-Box Model CSR, June 14, 2011
  63. 63. Intermediate Summary Want to understand tractability of a problem for general-purpose (randomized) search heuristics Query complexity as such is not a sufficient measure (Many) Randomized search heuristics do selection based on relative fitness values only, not on absolute values: Ranking-Based Black-Box Model Does it Mastermind problem help? 1 1 0 1 0 0 1 1 B. Doerr/C. Winzen: Ranking-Based Black-Box Model CSR, June 14, 2011
  64. 64. The Ranking-Based BBC of Mastermind is Θ(n / log n) n Mastermind problem 1 1 0 1 0 0 1 1 Classical Query Complexity n Θ( log n ) Search Heuristics Θ(n log n) Ranking-Based Query Complexity n Θ( log n ) B. Doerr/C. Winzen: Ranking-Based Black-Box Model CSR, June 14, 2011
  65. 65. Example: BinaryValue //Weighted Mastermind Carole chooses a binary string of length n and a permutation σ 1 1 0 1 0 0 1 1 24 22 21 23 25 28 26 27 σ=(4 2 1 3 5 8 6 7) Paul wants to find it. He may ask any string of length n: 1 0 1 0 1 0 1 0 Carole computes the weighted fitness value: 1 0 1 0 1 0 1 0 “Paul, your string has a score of 336 (=24+28+26)” B. Doerr/C. Winzen: Ranking-Based Black-Box Model CSR, June 14, 2011
  66. 66. The Query Complexity of BinaryValue is O(log n)Paul can do a binary search (parallel for each i≤n): 1 1 0 1 0 0 1 1 24 22 21 23 25 28 26 27 1 1 1 1 1 1 1 1 24+22+23+26+27 1 1 1 1 0 0 0 0 24+22+23+25+28 1 1 0 0 1 1 0 0 24+22+21 1 0 1 0 1 0 1 0 24+28+26 1 1 0 1 0 0 1 1 B. Doerr/C. Winzen: Ranking-Based Black-Box Model CSR, June 14, 2011
  67. 67. Binary Search not Possible in Ranking-Based ModelPaul can do a binary search (parallel for each i≤n): 1 1 0 1 0 0 1 1 24 22 21 23 25 28 26 27 1 1 1 1 1 1 1 1 28 1 1 1 1 0 0 0 0 317 1 1 0 0 1 1 0 0 -29 1 0 1 0 1 0 1 0 29 ? ? ? ? ? ? ? ? B. Doerr/C. Winzen: Ranking-Based Black-Box Model CSR, June 14, 2011
  68. 68. Binary Search not Possible in Ranking-Based ModelPaul can do a binary search (parallel for each i≤n): 1 1 0 1 0 0 1 1 24 22 21 23 25 28 26 27 1 1 1 1 1 1 1 1 28 1 1 1 1 0 0 0 0 317 In fact, 1 1 0 RBBBC(BinaryValue)=Θ(n) 0 1 1 0 0 -29 1 0 1 0 1 0 1 0 29 ? ? ? ? ? ? ? ? B. Doerr/C. Winzen: Ranking-Based Black-Box Model CSR, June 14, 2011
  69. 69. The Ranking-Based Black-Box Complexity of BinaryValue is Θ(n) n Limited If Algorithm queries two strings x and y, Learning it can learn at most 1 bit of the target string z. t=2 Example: g(BVz,σ (x)) > g(BVz,σ (y)) x=100000 ⇔ y=000000 z1 = x1 = 1 B. Doerr/C. Winzen: Ranking-Based Black-Box Model CSR, June 14, 2011
  70. 70. The Ranking-Based Black-Box Complexity of BinaryValue is Θ(n) n Limited If Algorithm queries two strings x and y, Learning it can learn at most 1 bit of the target string z. t=2 Limited If Algorithm queries t strings x1,...,xt, x Learning it can learn at most t-1 bit of the target string z. general t B. Doerr/C. Winzen: Ranking-Based Black-Box Model CSR, June 14, 2011
  71. 71. The Ranking-Based Black-Box Complexity of BinaryValue is Θ(n) n Limited If Algorithm queries two strings x and y, Learning it can learn at most 1 bit of the target string z. t=2 Limited If Algorithm queries t strings x1,...,xt, x Learning it can learn at most t-1 bit of the target string z. general t Ω(n) There is no deterministic algorithm which deterministic optimizes BinaryValue in sublinear time. lower bound B. Doerr/C. Winzen: Ranking-Based Black-Box Model CSR, June 14, 2011
  72. 72. The Ranking-Based Black-Box Complexity of BinaryValue is Θ(n) n Limited If Algorithm queries two strings x and y, Learning it can learn at most 1 bit of the target string z. t=2 Limited If Algorithm queries t strings x1,...,xt, x Learning it can learn at most t-1 bit of the target string z. general t Ω(n) There is no deterministic algorithm which deterministic optimizes BinaryValue in sublinear time. lower bound Ω(n) Follows from deterministic lower bound and randomized Yao’s minimax principle lower bound B. Doerr/C. Winzen: Ranking-Based Black-Box Model CSR, June 14, 2011
  73. 73. Summary Want to understand tractability of different problems for (randomized) search heuristics Measure: number of function evaluations B. Doerr/C. Winzen: Ranking-Based Black-Box Model CSR, June 14, 2011
  74. 74. Summary Want to understand tractability of different problems for (randomized) search heuristics Measure: number of function evaluations Our main interest are good lower bounds Classical query complexity: often too weak lower bounds B. Doerr/C. Winzen: Ranking-Based Black-Box Model CSR, June 14, 2011
  75. 75. Summary Want to understand tractability of different problems for (randomized) search heuristics Measure: number of function evaluations Our main interest are good lower bounds Classical query complexity: often too weak lower bounds Ranking-Based Black-Box Model: query complexity model only relative, not absolute fitness values are given B. Doerr/C. Winzen: Ranking-Based Black-Box Model CSR, June 14, 2011
  76. 76. Summary Want to understand tractability of different problems for (randomized) search heuristics Measure: number of function evaluations Our main interest are good lower bounds Classical query complexity: often too weak lower bounds Ranking-Based Black-Box Model: only relative, not absolute fitness values are given BinaryValue problem 11010011 Classical Query Complexity Θ(log n)Search Heuristics Θ(n log n) Ranking-Based Query Complexity Θ(n) B. Doerr/C. Winzen: Ranking-Based Black-Box Model CSR, June 14, 2011
  77. 77. Summary Want to understand tractability of different problems for (randomized) search heuristics Measure: number of function evaluations Our main interest are good lower bounds Classical query complexity: often too weak lower bounds Ranking-Based Black-Box Model: only relative, not absolute fitness values are given BinaryValue problem Mastermind problem 11010011 11010011 Classical Classical Query Complexity Query Complexity n Θ(log n) Θ( log n )Search Heuristics Search Heuristics Θ(n log n) Θ(n log n) Ranking-Based Ranking-Based Query Complexity Query Complexity n Θ(n) Θ( log n ) B. Doerr/C. Winzen: Ranking-Based Black-Box Model CSR, June 14, 2011
  78. 78. Future WorkPlenty! B. Doerr/C. Winzen: Ranking-Based Black-Box Model CSR, June 14, 2011
  79. 79. Future WorkPlenty! Other black-box models: restricted memory models unbiased sampling strategies combinations thereof ... B. Doerr/C. Winzen: Ranking-Based Black-Box Model CSR, June 14, 2011
  80. 80. Future WorkPlenty! Other black-box models: restricted memory models unbiased sampling strategies combinations thereof ... Combinatorial problems: MST, SSSP problems partition problem ... ... B. Doerr/C. Winzen: Ranking-Based Black-Box Model CSR, June 14, 2011
  81. 81. Future WorkPlenty! Other black-box models: restricted memory models unbiased sampling strategies Almost equivalent problem: combinations thereof n distinguishable balls of unknown weight ... 10 Combinatorial problems: 8 9 MST, SSSP problems 5 6 7 1 2 3 4 partition problem ... How often do you need to use the balance to find a perfect partition ... of the balls? B. Doerr/C. Winzen: Ranking-Based Black-Box Model CSR, June 14, 2011

×