Csr2011 june14 11_30_winzen

271 views

Published on

0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total views
271
On SlideShare
0
From Embeds
0
Number of Embeds
8
Actions
Shares
0
Downloads
2
Comments
0
Likes
0
Embeds 0
No embeds

No notes for slide

Csr2011 june14 11_30_winzen

  1. 1. Towards a Complexity Theory for Randomized Search Heuristics:The Ranking-Based Black-Box Model Benjamin Doerr / Carola Winzen CSR, June 14, 2011
  2. 2. Our Motivation General-purpose (randomized) search heuristics are ...easy to implement, ...very flexible, ...need little a priori knowledge about problem at hand, ...can deal with many constraints and nonlinearities, B. Doerr/C. Winzen: Ranking-Based Black-Box Model CSR, June 14, 2011
  3. 3. Our Motivation General-purpose (randomized) search heuristics are ...easy to implement, ...very flexible, ...need little a priori knowledge about problem at hand, ...can deal with many constraints and nonlinearities, and thus, frequently applied in practice. Some theoretical results exist. Typical result: runtime analysis for a particular algorithm on a particular problem. B. Doerr/C. Winzen: Ranking-Based Black-Box Model CSR, June 14, 2011
  4. 4. Our Motivation General-purpose (randomized) search heuristics are ...easy to implement, ...very flexible, ...need little a priori knowledge about problem at hand, ...can deal with many constraints and nonlinearities, and thus, frequently applied in practice. Some theoretical results exist. Typical result: runtime analysis for Comparable to a particular algorithm the “early days” on a particular problem. of Computer Science B. Doerr/C. Winzen: Ranking-Based Black-Box Model CSR, June 14, 2011
  5. 5. Our Motivation In classical theoretical computer science: first results: runtime analysis for a particular algorithm on a particular problem. B. Doerr/C. Winzen: Ranking-Based Black-Box Model CSR, June 14, 2011
  6. 6. Our Motivation In classical theoretical computer science: first results: runtime analysis for a particular algorithm on a particular problem. general lower bounds (“tractability of a problem”) complexity theory B. Doerr/C. Winzen: Ranking-Based Black-Box Model CSR, June 14, 2011
  7. 7. Our Motivation In classical theoretical computer science: first results: runtime analysis for a particular algorithm on a particular problem. general lower bounds (“tractability of a problem”) complexity theory How to create a complexity theory for randomized search heuristics? B. Doerr/C. Winzen: Ranking-Based Black-Box Model CSR, June 14, 2011
  8. 8. Our Motivation In classical theoretical computer science: first results: runtime analysis for a particular algorithm on a particular problem. general lower bounds (“tractability of a problem”) complexity theory Our aim: to understand the tractability of a problem for general-purpose (randomized) search heuristics “Towards a Complexity Theory for Randomized Search Heuristics” B. Doerr/C. Winzen: Ranking-Based Black-Box Model CSR, June 14, 2011
  9. 9. A General View on Search Heuristics Search Heuristic f Black-Box = “Oracle” B. Doerr/C. Winzen: Ranking-Based Black-Box Model CSR, June 14, 2011
  10. 10. A General View on Search Heuristics x Search Heuristic f Black-Box = “Oracle” B. Doerr/C. Winzen: Ranking-Based Black-Box Model CSR, June 14, 2011
  11. 11. A General View on Search Heuristics x Search Heuristic f f(x) x Black-Box = “Oracle” B. Doerr/C. Winzen: Ranking-Based Black-Box Model CSR, June 14, 2011
  12. 12. A General View on Search Heuristics x Search Heuristic f f(x) x Black-Box = “Oracle” Typical cost measure: number of function evaluations until an optimal solution is queried for the first time B. Doerr/C. Winzen: Ranking-Based Black-Box Model CSR, June 14, 2011
  13. 13. A General View on Search Heuristics Classical Query Complexity Model x Search Heuristic f f(x) x Black-Box = “Oracle” Typical cost measure: number of function evaluations until an optimal solution is queried for the first time B. Doerr/C. Winzen: Ranking-Based Black-Box Model CSR, June 14, 2011
  14. 14. A Theory for (Randomized) Search Heuristics Part 1: Classical query complexity model Game theoretic view Example: Mastermind Part 2: Refinement: ranking-based query complexity “Towards a Complexity Theory for Randomized Search Heuristics: The Ranking-Based Black-Box Model” B. Doerr/C. Winzen: Ranking-Based Black-Box Model CSR, June 14, 2011
  15. 15. Example: A Mastermind Problem Carole (=oracle) chooses a binary string of length n: 1 1 0 1 0 0 1 1 B. Doerr/C. Winzen: Ranking-Based Black-Box Model CSR, June 14, 2011
  16. 16. Example: A Mastermind Problem Carole (=oracle) chooses a binary string of length n: 1 1 0 1 0 0 1 1 Paul (=algo.) tries to find it. He may ask any string of length n: B. Doerr/C. Winzen: Ranking-Based Black-Box Model CSR, June 14, 2011
  17. 17. Example: A Mastermind Problem Carole (=oracle) chooses a binary string of length n: 1 1 0 1 0 0 1 1 Paul (=algo.) tries to find it. He may ask any string of length n: 1 0 1 0 1 0 1 0 B. Doerr/C. Winzen: Ranking-Based Black-Box Model CSR, June 14, 2011
  18. 18. Example: A Mastermind Problem Carole (=oracle) chooses a binary string of length n: 1 1 0 1 0 0 1 1 Paul (=algo.) tries to find it. He may ask any string of length n: 1 0 1 0 1 0 1 0 Carole computes in how many positions the strings coincide: B. Doerr/C. Winzen: Ranking-Based Black-Box Model CSR, June 14, 2011
  19. 19. Example: A Mastermind Problem Carole (=oracle) chooses a binary string of length n: 1 1 0 1 0 0 1 1 Paul (=algo.) tries to find it. He may ask any string of length n: 1 0 1 0 1 0 1 0 Carole computes in how many positions the strings coincide: 1 0 1 0 1 0 1 0 B. Doerr/C. Winzen: Ranking-Based Black-Box Model CSR, June 14, 2011
  20. 20. Example: A Mastermind Problem Carole (=oracle) chooses a binary string of length n: 1 1 0 1 0 0 1 1 Paul (=algo.) tries to find it. He may ask any string of length n: 1 0 1 0 1 0 1 0 Carole computes in how many positions the strings coincide: 1 0 1 0 1 0 1 0 “Paul, our strings coincide in 3 bits” B. Doerr/C. Winzen: Ranking-Based Black-Box Model CSR, June 14, 2011
  21. 21. Example: A Mastermind Problem Carole (=oracle) chooses a binary string of length n: 1 1 0 1 0 0 1 1 Paul (=algo.) tries to find it. He may ask any string of length n: 1 0 1 0 1 0 1 0 Carole computes in how many positions the strings coincide: 1 0 1 0 1 0 1 0 We say that the “Paul, our strings coincide in 3 bits” “fitness” of Paul’s string is 3 B. Doerr/C. Winzen: Ranking-Based Black-Box Model CSR, June 14, 2011
  22. 22. Example: A Mastermind Problem Carole chooses a binary string of length n: 1 1 0 1 0 0 1 1 Paul tries to find it. He may ask any string of length n: 1 0 1 0 1 0 1 0 3 B. Doerr/C. Winzen: Ranking-Based Black-Box Model CSR, June 14, 2011
  23. 23. Example: A Mastermind Problem Carole chooses a binary string of length n: 1 1 0 1 0 0 1 1 Paul tries to find it. He may ask any string of length n: 1 0 1 0 1 0 1 0 3 1 0 1 1 0 1 0 1 B. Doerr/C. Winzen: Ranking-Based Black-Box Model CSR, June 14, 2011
  24. 24. Example: A Mastermind Problem Carole chooses a binary string of length n: 1 1 0 1 0 0 1 1 Paul tries to find it. He may ask any string of length n: 1 0 1 0 1 0 1 0 3 1 0 1 1 0 1 0 1 B. Doerr/C. Winzen: Ranking-Based Black-Box Model CSR, June 14, 2011
  25. 25. Example: A Mastermind Problem Carole chooses a binary string of length n: 1 1 0 1 0 0 1 1 Paul tries to find it. He may ask any string of length n: 1 0 1 0 1 0 1 0 3 1 0 1 1 0 1 0 1 4 ... B. Doerr/C. Winzen: Ranking-Based Black-Box Model CSR, June 14, 2011
  26. 26. Example: A Mastermind Problem Carole chooses a binary string of length n: 1 1 0 1 0 0 1 1 Paul tries to find it. He may ask any string of length n: 1 0 1 0 1 0 1 0 3 1 0 1 1 0 1 0 1 4 ... How many queries does Paul need, on average, until he has identified Carole’s string? B. Doerr/C. Winzen: Ranking-Based Black-Box Model CSR, June 14, 2011
  27. 27. Reminder Our aim: To understand tractability of a problem for general-purpose (randomized) search heuristics Measure: number of function evaluations until an optimal solution is queried for the first time Our main interest: good lower bounds B. Doerr/C. Winzen: Ranking-Based Black-Box Model CSR, June 14, 2011
  28. 28. The Master Mind Problem: What Search Heuristics Do Paul tries to find Carole’s binary string of length n: 1 1 0 1 0 0 1 1 B. Doerr/C. Winzen: Ranking-Based Black-Box Model CSR, June 14, 2011
  29. 29. The Master Mind Problem: What Search Heuristics Do Paul tries to find Carole’s binary string of length n: 1 1 0 1 0 0 1 1 First query is arbitrary: 1 0 1 0 1 0 1 0 B. Doerr/C. Winzen: Ranking-Based Black-Box Model CSR, June 14, 2011
  30. 30. The Master Mind Problem: What Search Heuristics Do Paul tries to find Carole’s binary string of length n: 1 1 0 1 0 0 1 1 First query is arbitrary: 1 0 1 0 1 0 1 0 3 B. Doerr/C. Winzen: Ranking-Based Black-Box Model CSR, June 14, 2011
  31. 31. The Master Mind Problem: What Search Heuristics Do Paul tries to find Carole’s binary string of length n: 1 1 0 1 0 0 1 1 First query is arbitrary: 1 0 1 0 1 0 1 0 3 Then flip exactly one bit (chosen u.a.r.): 1 1 1 0 1 0 1 0 B. Doerr/C. Winzen: Ranking-Based Black-Box Model CSR, June 14, 2011
  32. 32. The Master Mind Problem: What Search Heuristics Do Paul tries to find Carole’s binary string of length n: 1 1 0 1 0 0 1 1 First query is arbitrary: 1 0 1 0 1 0 1 0 3 Then flip exactly one bit (chosen u.a.r.): 1 1 1 0 1 0 1 0 4 B. Doerr/C. Winzen: Ranking-Based Black-Box Model CSR, June 14, 2011
  33. 33. The Master Mind Problem: What Search Heuristics Do Paul tries to find Carole’s binary string of length n: 1 1 0 1 0 0 1 1 First query is arbitrary: 1 0 1 0 1 0 1 0 3 Then flip exactly one bit (chosen u.a.r.): 1 1 1 0 1 0 1 0 4 And it continues with the better of the two: 1 1 1 0 1 1 1 0 B. Doerr/C. Winzen: Ranking-Based Black-Box Model CSR, June 14, 2011
  34. 34. The Master Mind Problem: What Search Heuristics Do Paul tries to find Carole’s binary string of length n: 1 1 0 1 0 0 1 1 First query is arbitrary: 1 0 1 0 1 0 1 0 3 Then flip exactly one bit (chosen u.a.r.): 1 1 1 0 1 0 1 0 4 Random Local Search algorithm: Θ(n log n) Coupon Collector [Folklore] B. Doerr/C. Winzen: Ranking-Based Black-Box Model CSR, June 14, 2011
  35. 35. The Master Mind Problem: What Search Heuristics Do Paul tries to find Carole’s binary string of length n: 1 1 0 1 0 0 1 1 First query is arbitrary: 1 0 1 0 1 0 1 0 B. Doerr/C. Winzen: Ranking-Based Black-Box Model CSR, June 14, 2011
  36. 36. The Master Mind Problem: What Search Heuristics Do Paul tries to find Carole’s binary string of length n: 1 1 0 1 0 0 1 1 First query is arbitrary: 1 0 1 0 1 0 1 0 3 Flip each bit with probability 1/n: B. Doerr/C. Winzen: Ranking-Based Black-Box Model CSR, June 14, 2011
  37. 37. The Master Mind Problem: What Search Heuristics Do Paul tries to find Carole’s binary string of length n: 1 1 0 1 0 0 1 1 First query is arbitrary: 1 0 1 0 1 0 1 0 3 Flip each bit with probability 1/n: 0 0 1 0 1 1 1 0 1 B. Doerr/C. Winzen: Ranking-Based Black-Box Model CSR, June 14, 2011
  38. 38. The Master Mind Problem: What Search Heuristics Do Paul tries to find Carole’s binary string of length n: 1 1 0 1 0 0 1 1 First query is arbitrary: 1 0 1 0 1 0 1 0 3 Flip each bit with probability 1/n: 0 0 1 0 1 1 1 0 1 And it continues with the better of the two B. Doerr/C. Winzen: Ranking-Based Black-Box Model CSR, June 14, 2011
  39. 39. The Master Mind Problem: What Search Heuristics Do Paul tries to find Carole’s binary string of length n: 1 1 0 1 0 0 1 1 First query is arbitrary: 1 0 1 0 1 0 1 0 3 Flip each bit with probability 1/n: 0 0 1 0 1 1 1 0 1 (1+1) Evolutionary Algorithm: [Mühlenbein Θ(n log n)[Droste/Jansen/Wegener 92] 02] B. Doerr/C. Winzen: Ranking-Based Black-Box Model CSR, June 14, 2011
  40. 40. The Master Mind Problem: Optimal Strategies (1/2) Paul tries to find Carole’s binary string of length n: 1 0 0 1 0 0 1 1 B. Doerr/C. Winzen: Ranking-Based Black-Box Model CSR, June 14, 2011
  41. 41. The Master Mind Problem: Optimal Strategies (1/2) Paul tries to find Carole’s binary string of length n: 1 0 0 1 0 0 1 1 He can go through the string bit by bit: 0 0 0 0 0 0 0 0 4 B. Doerr/C. Winzen: Ranking-Based Black-Box Model CSR, June 14, 2011
  42. 42. The Master Mind Problem: Optimal Strategies (1/2) Paul tries to find Carole’s binary string of length n: 1 0 0 1 0 0 1 1 He can go through the string bit by bit: 0 0 0 0 0 0 0 0 4 1 0 0 0 0 0 0 0 5 B. Doerr/C. Winzen: Ranking-Based Black-Box Model CSR, June 14, 2011
  43. 43. The Master Mind Problem: Optimal Strategies (1/2) Paul tries to find Carole’s binary string of length n: 1 0 0 1 0 0 1 1 He can go through the string bit by bit: 0 0 0 0 0 0 0 0 4 1 0 0 0 0 0 0 0 5 1 1 0 0 0 0 0 0 4 B. Doerr/C. Winzen: Ranking-Based Black-Box Model CSR, June 14, 2011
  44. 44. The Master Mind Problem: Optimal Strategies (1/2) Paul tries to find Carole’s binary string of length n: 1 0 0 1 0 0 1 1 He can go through the string bit by bit: 0 0 0 0 0 0 0 0 4 1 0 0 0 0 0 0 0 5 1 1 0 0 0 0 0 0 4 ... 1 0 0 1 0 0 1 1 8 B. Doerr/C. Winzen: Ranking-Based Black-Box Model CSR, June 14, 2011
  45. 45. The Master Mind Problem: Optimal Strategies (1/2) Paul tries to find Carole’s binary string of length n: 1 0 0 1 0 0 1 1 He can go through the string bit by bit: 0 0 0 0 0 0 0 0 4 1 0 0 0 0 0 0 0 5 1 1 0 0 0 0 0 0 4 O(n) ... 1 0 0 1 0 0 1 1 8 B. Doerr/C. Winzen: Ranking-Based Black-Box Model CSR, June 14, 2011
  46. 46. The Master Mind Problem: Optimal Strategies (1/2) Paul tries to find Carole’s binary string of length n: 1 0 0 1 0 0 1 1 He can go through the string bit by bit: 0 0 0 0 0 0 0 0 4 1 0 0 0 0 0 0 0 5 1 1 0 0 0 0 0 0 4 O(n) ... Can we do 1 0 0 1 0 0 1 1 even8 better? B. Doerr/C. Winzen: Ranking-Based Black-Box Model CSR, June 14, 2011
  47. 47. The Master Mind Problem: Optimal Strategies (2/2) 1 1 0 1 0 0 1 1 B. Doerr/C. Winzen: Ranking-Based Black-Box Model CSR, June 14, 2011
  48. 48. The Master Mind Problem: Optimal Strategies (2/2) 1 1 0 1 0 0 1 1 1 1 1 1 0 1 0 0 1 4 2 0 1 0 1 0 0 0 0 5 cn 0 0 0 1 1 1 1 1 4log n B. Doerr/C. Winzen: Ranking-Based Black-Box Model CSR, June 14, 2011
  49. 49. 0 1 1 1 Optimal Strategies (2/2) 0 0 1 1 0 0 0 1 0 0 1 0 1 1 0 1 0 0 1 1 0 0 1 1 0 0 0 1 0 0 1 1 1 1 1 1 0 1 0 0 1 4 0 0 1 1 2 0 1 0 1 0 0 0 0 5 1 1 ... 1 3 4 . . 4 . 5 0 1 ... 0 6 5 . . 5 . 2 cn 0 0 ... 1 3 4 . . 4 . 5 0 0 0 1 1 1 1 1 4log n B. Doerr/C. Winzen: Ranking-Based Black-Box Model CSR, June 14, 2011
  50. 50. 0 1 1 1 Optimal Strategies (2/2) 0 0 1 1 0 0 0 1 0 0 1 0 1 1 0 1 0 0 1 1 0 0 1 1 0 0 0 1 0 0 1 1 1 1 1 1 0 1 0 0 1 4 0 0 1 1 2 0 1 0 1 0 0 0 0 5 1 1 ... 1 3 4 . . 4 . 5 0 1 ... 0 6 5 . . 5 . 2 cn 0 0 ... 1 3 4 . . 4 . 5 0 0 0 1 1 1 1 1 4log n B. Doerr/C. Winzen: Ranking-Based Black-Box Model CSR, June 14, 2011
  51. 51. 0 1 1 1 Optimal Strategies (2/2) 0 0 1 1 0 0 0 1 0 0 1 0 1 1 0 1 0 0 1 1 0 0 1 1 0 0 0 1 0 0 1 1 1 1 1 1 0 1 0 0 1 4 0 0 1 1 2 0 1 0 1 0 0 0 0 5 1 1 ... 1 3 4 . . 4 . 5 0 1 ... 0 6 5 . . 5 . 2 cn 0 0 ... 1 3 4 . . 4 . 5 0 0 0 1 1 1 1 1 4log n B. Doerr/C. Winzen: Ranking-Based Black-Box Model CSR, June 14, 2011
  52. 52. 0 1 1 1 Optimal Strategies (2/2) 0 0 1 1 0 0 0 1 0 0 1 0 1 1 0 1 0 0 1 1 0 0 1 1 0 0 0 1 0 0 1 1 1 1 1 1 0 1 0 0 1 4 0 0 1 1 2 0 1 0 1 0 0 0 0 5 1 1 ... 1 3 4 . . 4 . 5 0 1 ... 0 6 5 . . 5 . 2 cn 0 0 ... 1 3 4 . . 4 . 5 0 0 0 1 1 1 1 1 4log n B. Doerr/C. Winzen: Ranking-Based Black-Box Model CSR, June 14, 2011
  53. 53. 0 1 1 1 Optimal Strategies (2/2) 0 0 1 1 0 0 0 1 0 0 1 0 1 1 0 1 0 0 1 1 0 0 1 1 0 0 0 1 0 0 1 1 1 1 1 1 0 1 0 0 1 4 0 0 1 1 2 0 1 0 1 0 0 0 0 5 1 1 ... 1 3 4 . . 4 . 5 0 1 ... 0 6 5 . . 5 . 2 Fitness elimination technique n O( log n ) cn 0 0 ... 1 3 4 . . 4 . 5 0 0 0 1 1 1 1 1 4log n [Anil/Wiegand 09], see also [D./Johannsen/Kötzing/Lehre/Wagner/W. 11] B. Doerr/C. Winzen: Ranking-Based Black-Box Model CSR, June 14, 2011
  54. 54. 0 1 1 1 Optimal Strategies (2/2) 0 0 1 1 0 0 0 1 0 0 1 0 1 1 0 1 0 0 1 1 0 0 1 1 0 0 0 1 0 0 1 1 1 1 1 1 0 1 0 0 1 4 0 0 1 1 2 0 1 0 1 0 0 0 0 5 1 1 ... 1 3 4 . . 4 . 5 0 1 ... 0 6 5 . . 5 . 2 Fitness elimination technique n Θ( log n ) cn 0 0 ... 1 3 4 . . 4 . 5 0 0 0 1 1 1 1 1 4log n [Anil/Wiegand 09], see also [D./Johannsen/Kötzing/Lehre/Wagner/W. 11] B. Doerr/C. Winzen: Ranking-Based Black-Box Model CSR, June 14, 2011
  55. 55. Intermediate Summary Want to understand tractability of a problem for general- purpose (randomized) search heuristics Query complexity as such is not a sufficient measure: Mastermind problem 1 1 0 1 0 0 1 1 Search Heuristics Query Complexity n Θ(n log n) Θ( log n ) B. Doerr/C. Winzen: Ranking-Based Black-Box Model CSR, June 14, 2011
  56. 56. The Ranking-Based Black-Box Model Observation: many randomized search heuristics use fitness values only to compare B. Doerr/C. Winzen: Ranking-Based Black-Box Model CSR, June 14, 2011
  57. 57. The Ranking-Based Black-Box Model Observation: many randomized search heuristics use fitness values only to compare RLS (1+1) EA ... B. Doerr/C. Winzen: Ranking-Based Black-Box Model CSR, June 14, 2011
  58. 58. The Ranking-Based Black-Box Model Observation: many randomized search heuristics use fitness values only to compare RLS (1+1) EA ... B. Doerr/C. Winzen: Ranking-Based Black-Box Model CSR, June 14, 2011
  59. 59. The Ranking-Based Black-Box ModelDoes not reveal absolute fitness values: Algorithm f Black-Box = “Oracle” B. Doerr/C. Winzen: Ranking-Based Black-Box Model CSR, June 14, 2011
  60. 60. The Ranking-Based Black-Box ModelDoes not reveal absolute fitness values: x Algorithm f Black-Box = “Oracle” B. Doerr/C. Winzen: Ranking-Based Black-Box Model CSR, June 14, 2011
  61. 61. The Ranking-Based Black-Box ModelDoes not reveal absolute fitness values: x Algorithm f Rank 1 x Rank 2 x Black-Box Rank 2 x = “Oracle” Rank 4 x B. Doerr/C. Winzen: Ranking-Based Black-Box Model CSR, June 14, 2011
  62. 62. The Ranking-Based Black-Box ModelEquivalent formulation: Let g : R → R be a strictly monotone function x Algorithm f g(f(x))=127 Rank 1 g g(f(x)) fx g(f(x))=125 Rank 2 Black-Box g(f(x))=125 Rank 2 g(f(x))=27 Rank 4 = “Oracle” B. Doerr/C. Winzen: Ranking-Based Black-Box Model CSR, June 14, 2011
  63. 63. Intermediate Summary Want to understand tractability of a problem for general-purpose (randomized) search heuristics Query complexity as such is not a sufficient measure (Many) Randomized search heuristics do selection based on relative fitness values only, not on absolute values: Ranking-Based Black-Box Model es it Mastermind problem Do help? 1 1 0 1 0 0 1 1 B. Doerr/C. Winzen: Ranking-Based Black-Box Model CSR, June 14, 2011
  64. 64. nThe Ranking-Based BBC of Mastermind is Θ(n / log n) Mastermind problem 1 1 0 1 0 0 1 1 Classical Query Complexity n Θ( log n ) Search Heuristics Θ(n log n) Ranking-Based Query Complexity n Θ( log n ) B. Doerr/C. Winzen: Ranking-Based Black-Box Model CSR, June 14, 2011
  65. 65. Example: BinaryValue //Weighted Mastermind Carole chooses a binary string of length n and a permutation σ 1 1 0 1 0 0 1 1 24 22 21 23 25 28 26 27 σ=(4 2 1 3 5 8 6 7) Paul wants to find it. He may ask any string of length n: 1 0 1 0 1 0 1 0 Carole computes the weighted fitness value: 1 0 1 0 1 0 1 0 “Paul, your string has a score of 336 (=24+28+26)” B. Doerr/C. Winzen: Ranking-Based Black-Box Model CSR, June 14, 2011
  66. 66. The Query Complexity of BinaryValue is O(log n)Paul can do a binary search (parallel for each i≤n): 1 1 0 1 0 0 1 1 24 22 21 23 25 28 26 27 1 1 1 1 1 1 1 1 24+22+23+26+27 1 1 1 1 0 0 0 0 24+22+23+25+28 1 1 0 0 1 1 0 0 24+22+21 1 0 1 0 1 0 1 0 24+28+26 1 1 0 1 0 0 1 1 B. Doerr/C. Winzen: Ranking-Based Black-Box Model CSR, June 14, 2011
  67. 67. Binary Search not Possible in Ranking-Based ModelPaul can do a binary search (parallel for each i≤n): 1 1 0 1 0 0 1 1 24 22 21 23 25 28 26 27 1 1 1 1 1 1 1 1 28 1 1 1 1 0 0 0 0 317 1 1 0 0 1 1 0 0 -29 1 0 1 0 1 0 1 0 29 ? ? ? ? ? ? ? ? B. Doerr/C. Winzen: Ranking-Based Black-Box Model CSR, June 14, 2011
  68. 68. Binary Search not Possible in Ranking-Based ModelPaul can do a binary search (parallel for each i≤n): 1 1 0 1 0 0 1 1 24 22 21 23 25 28 26 27 1 1 1 1 1 1 1 1 28 1 1 1 1 0 0 0 0 317 In fact, 1 1 -29 0 RBBBC(BinaryValue)=Θ(n) 0 1 1 0 0 1 0 1 0 1 0 1 0 29 ? ? ? ? ? ? ? ? B. Doerr/C. Winzen: Ranking-Based Black-Box Model CSR, June 14, 2011
  69. 69. The Ranking-Based Black-Box Complexity of BinaryValue is Θ(n) n Limited If Algorithm queries two strings x and y, Learning it can learn at most 1 bit of the target string z. t=2 Example: g(BVz,σ (x)) > g(BVz,σ (y)) x=100000 ⇔ y=000000 z1 = x1 = 1 B. Doerr/C. Winzen: Ranking-Based Black-Box Model CSR, June 14, 2011
  70. 70. The Ranking-Based Black-Box Complexity of BinaryValue is Θ(n) n Limited If Algorithm queries two strings x and y, Learning it can learn at most 1 bit of the target string z. t=2 Limited x If Algorithm queries t strings x1,...,xt, Learning it can learn at most t-1 bit of the target string z. general t B. Doerr/C. Winzen: Ranking-Based Black-Box Model CSR, June 14, 2011
  71. 71. The Ranking-Based Black-Box Complexity of BinaryValue is Θ(n) n Limited If Algorithm queries two strings x and y, Learning it can learn at most 1 bit of the target string z. t=2 Limited x If Algorithm queries t strings x1,...,xt, Learning it can learn at most t-1 bit of the target string z. general t Ω(n) There is no deterministic algorithm which deterministic optimizes BinaryValue in sublinear time. lower bound B. Doerr/C. Winzen: Ranking-Based Black-Box Model CSR, June 14, 2011
  72. 72. The Ranking-Based Black-Box Complexity of BinaryValue is Θ(n) n Limited If Algorithm queries two strings x and y, Learning it can learn at most 1 bit of the target string z. t=2 Limited x If Algorithm queries t strings x1,...,xt, Learning it can learn at most t-1 bit of the target string z. general t Ω(n) There is no deterministic algorithm which deterministic optimizes BinaryValue in sublinear time. lower bound Ω(n) Follows from deterministic lower bound and randomized Yao’s minimax principle lower bound B. Doerr/C. Winzen: Ranking-Based Black-Box Model CSR, June 14, 2011
  73. 73. Summary Want to understand tractability of different problems for (randomized) search heuristics Measure: number of function evaluations B. Doerr/C. Winzen: Ranking-Based Black-Box Model CSR, June 14, 2011
  74. 74. Summary Want to understand tractability of different problems for (randomized) search heuristics Measure: number of function evaluations Our main interest are good lower bounds Classical query complexity: often too weak lower bounds B. Doerr/C. Winzen: Ranking-Based Black-Box Model CSR, June 14, 2011
  75. 75. Summary Want to understand tractability of different problems for (randomized) search heuristics Measure: number of function evaluations Our main interest are good lower bounds Classical query complexity: often too weak lower bounds Ranking-Based Black-Box Model: query complexity model only relative, not absolute fitness values are given B. Doerr/C. Winzen: Ranking-Based Black-Box Model CSR, June 14, 2011
  76. 76. Summary Want to understand tractability of different problems for (randomized) search heuristics Measure: number of function evaluations Our main interest are good lower bounds Classical query complexity: often too weak lower bounds Ranking-Based Black-Box Model: only relative, not absolute fitness values are given BinaryValue problem 11010011 Classical Query Complexity Θ(log n)Search Heuristics Θ(n log n) Ranking-Based Query Complexity Θ(n) B. Doerr/C. Winzen: Ranking-Based Black-Box Model CSR, June 14, 2011
  77. 77. Summary Want to understand tractability of different problems for (randomized) search heuristics Measure: number of function evaluations Our main interest are good lower bounds Classical query complexity: often too weak lower bounds Ranking-Based Black-Box Model: only relative, not absolute fitness values are given BinaryValue problem Mastermind problem 11010011 11010011 Classical Classical Query Complexity Query Complexity n Θ(log n) Θ( log n )Search Heuristics Search Heuristics Θ(n log n) Θ(n log n) Ranking-Based Ranking-Based Query Complexity Query Complexity n Θ(n) Θ( log n ) B. Doerr/C. Winzen: Ranking-Based Black-Box Model CSR, June 14, 2011
  78. 78. Future WorkPlenty! B. Doerr/C. Winzen: Ranking-Based Black-Box Model CSR, June 14, 2011
  79. 79. Future WorkPlenty! Other black-box models: restricted memory models unbiased sampling strategies combinations thereof ... B. Doerr/C. Winzen: Ranking-Based Black-Box Model CSR, June 14, 2011
  80. 80. Future WorkPlenty! Other black-box models: restricted memory models unbiased sampling strategies combinations thereof ... Combinatorial problems: MST, SSSP problems partition problem ... ... B. Doerr/C. Winzen: Ranking-Based Black-Box Model CSR, June 14, 2011
  81. 81. Future WorkPlenty! Other black-box models: restricted memory models unbiased sampling strategies Almost equivalent problem: combinations thereof n distinguishable balls of unknown weight ... 10 Combinatorial problems: 8 9 MST, SSSP problems 5 6 7 1 2 3 4 partition problem ... How often do you need to use the balance to find a perfect partition ... of the balls? B. Doerr/C. Winzen: Ranking-Based Black-Box Model CSR, June 14, 2011

×