The document discusses creating a complexity theory for randomized search heuristics. It uses the example of the Mastermind problem, where an oracle chooses a secret binary string and an algorithm tries to discover it by querying strings and receiving feedback on matches. This is modeled as a black-box optimization problem. The document proposes analyzing such problems using a ranking-based query complexity model rather than only analyzing specific algorithms on specific problems. It suggests this approach could provide general lower bounds and help develop a complexity theory for randomized search heuristics.
Analogy is one of the most studied representatives of a family of non-classical forms of reasoning working across different domains, usually taken to play a crucial role in creative thought and problem-solving. In the first part of the talk, I will shortly introduce general principles of computational analogy models (relying on a generalization-based approach to analogy-making). We will then have a closer look at Heuristic-Driven Theory Projection (HDTP) as an example for a theoretical framework and implemented system: HDTP computes analogical relations and inferences for domains which are represented using many-sorted first-order logic languages, applying a restricted form of higher-order anti-unification for finding shared structural elements common to both domains. The presentation of the framework will be followed by a few reflections on the "cognitive plausibility" of the approach motivated by theoretical complexity and tractability considerations.
In the second part of the talk I will discuss an application of HDTP to modeling essential parts of concept blending processes as current "hot topic" in Cognitive Science. Here, I will sketch an analogy-inspired formal account of concept blending —developed in the European FP7-funded Concept Invention Theory (COINVENT) project— combining HDTP with mechanisms from Case-Based Reasoning.
Kristoffer Arnsfelt Hansen, Rasmus Ibsen-Jensen and Peter Bro Miltersen. The complexity of solving reachability games using value and strategy iteration
This document discusses the relationships between orbits of linear maps and regular languages. It shows that the chamber hitting problem (CHP) and permutation filter-realizability problem are Turing equivalent. It also shows that the injective filter-realizability problem and surjective filter-realizability problem are decidable, while the track product of the periodic and permutation filter-realizability problem is undecidable. The zero in the upper right corner problem, which is undecidable, can be reduced to the latter regular realizability problem.
The document describes a method for canonizing graphs of bounded treewidth in AC1 complexity. It presents the following:
1) Existing results showing canonization of bounded treewidth graphs is in P, TC1, TC2, and LogCFL complexity classes.
2) A new algorithm that canonizes bounded treewidth graphs in AC1 complexity by computing a tree decomposition of depth O(log n) and constructing a minimal description circuit of depth O(log n).
3) The algorithm works by computing descriptions for bags of the tree decomposition in parallel, sorting descriptions, and recursively combining descriptions while maintaining a circuit depth of O(log n).
This document summarizes research on the combinatorial properties of Burrows-Wheeler Transforms (BWT). It discusses prior work that characterized words with simple BWT image forms. It also introduces two general decision problems about BWT images and claims to provide efficient solutions to these problems. Specifically, it presents a theorem providing a criterion to check whether a given word is a valid BWT image based on analyzing the number of orbits in the word's stable sorting.
This document advocates against violence towards women and encourages women in abusive relationships to seek help. It warns women not to wait for an abusive situation to change as it will likely continue or escalate. Women are urged to not be silent and to scream for help if they are being tortured or made to feel like a slave. The overall message is one of empowering women in abusive situations to stand up for themselves, seek support from others, and remove themselves from relationships where they are being physically or psychologically harmed.
The document summarizes the musical history and collaborations between Eric Clapton and members of Derek and the Dominos. It mentions that Eric Clapton, Bobby Whitlock, Carl Radle, and Jim Gordon formed Derek and the Dominos in 1971. Their album Layla and Other Assorted Love Songs featured the hit song "Layla" and was influenced by Duane Allman of The Allman Brothers Band.
Analogy is one of the most studied representatives of a family of non-classical forms of reasoning working across different domains, usually taken to play a crucial role in creative thought and problem-solving. In the first part of the talk, I will shortly introduce general principles of computational analogy models (relying on a generalization-based approach to analogy-making). We will then have a closer look at Heuristic-Driven Theory Projection (HDTP) as an example for a theoretical framework and implemented system: HDTP computes analogical relations and inferences for domains which are represented using many-sorted first-order logic languages, applying a restricted form of higher-order anti-unification for finding shared structural elements common to both domains. The presentation of the framework will be followed by a few reflections on the "cognitive plausibility" of the approach motivated by theoretical complexity and tractability considerations.
In the second part of the talk I will discuss an application of HDTP to modeling essential parts of concept blending processes as current "hot topic" in Cognitive Science. Here, I will sketch an analogy-inspired formal account of concept blending —developed in the European FP7-funded Concept Invention Theory (COINVENT) project— combining HDTP with mechanisms from Case-Based Reasoning.
Kristoffer Arnsfelt Hansen, Rasmus Ibsen-Jensen and Peter Bro Miltersen. The complexity of solving reachability games using value and strategy iteration
This document discusses the relationships between orbits of linear maps and regular languages. It shows that the chamber hitting problem (CHP) and permutation filter-realizability problem are Turing equivalent. It also shows that the injective filter-realizability problem and surjective filter-realizability problem are decidable, while the track product of the periodic and permutation filter-realizability problem is undecidable. The zero in the upper right corner problem, which is undecidable, can be reduced to the latter regular realizability problem.
The document describes a method for canonizing graphs of bounded treewidth in AC1 complexity. It presents the following:
1) Existing results showing canonization of bounded treewidth graphs is in P, TC1, TC2, and LogCFL complexity classes.
2) A new algorithm that canonizes bounded treewidth graphs in AC1 complexity by computing a tree decomposition of depth O(log n) and constructing a minimal description circuit of depth O(log n).
3) The algorithm works by computing descriptions for bags of the tree decomposition in parallel, sorting descriptions, and recursively combining descriptions while maintaining a circuit depth of O(log n).
This document summarizes research on the combinatorial properties of Burrows-Wheeler Transforms (BWT). It discusses prior work that characterized words with simple BWT image forms. It also introduces two general decision problems about BWT images and claims to provide efficient solutions to these problems. Specifically, it presents a theorem providing a criterion to check whether a given word is a valid BWT image based on analyzing the number of orbits in the word's stable sorting.
This document advocates against violence towards women and encourages women in abusive relationships to seek help. It warns women not to wait for an abusive situation to change as it will likely continue or escalate. Women are urged to not be silent and to scream for help if they are being tortured or made to feel like a slave. The overall message is one of empowering women in abusive situations to stand up for themselves, seek support from others, and remove themselves from relationships where they are being physically or psychologically harmed.
The document summarizes the musical history and collaborations between Eric Clapton and members of Derek and the Dominos. It mentions that Eric Clapton, Bobby Whitlock, Carl Radle, and Jim Gordon formed Derek and the Dominos in 1971. Their album Layla and Other Assorted Love Songs featured the hit song "Layla" and was influenced by Duane Allman of The Allman Brothers Band.
The document discusses computational models for algebraic decision trees and algebraic computation trees over a ground field F. It describes how algebraic decision trees use polynomials of degree ≤ d to branch at each node, while algebraic computation trees allow testing polynomials to be calculated from previous polynomials along the path. The document then covers existing lower bounds on the complexity C(S) of the membership problem for a set S in terms of topological invariants of S, such as the number of connected components, Euler characteristic, and sum of Betti numbers.
The document discusses recognizing sparse perfect elimination bipartite graphs. It begins with an example of Gaussian elimination on a matrix that introduces new non-zero values. The key points are that perfect elimination bipartite graphs correspond to matrices that can be eliminated without creating new non-zeros, and this can be achieved by finding a sequence of bisimplicial edges in the corresponding bipartite graph. The document proposes using bisimplicial edges as pivots during elimination to avoid introducing new non-zeros.
The document discusses recognizing sparse perfect elimination bipartite graphs through matrix elimination. It provides an example of Gaussian elimination on a matrix that introduces new non-zero values. The key points are:
- Perfect elimination bipartite graphs correspond to matrices that allow elimination without creating new non-zeros.
- Existing algorithms have time complexity of O(n^5) or O(n^3/log n) but may produce dense matrices from sparse ones.
- A new algorithm is proposed that avoids this issue by working directly with the sparse matrix structure.
The document discusses the method of multiplicities, which is a technique for combinatorics using algebra. It involves finding a polynomial that vanishes on a set with high multiplicity. This is applied to problems in list decoding of Reed-Solomon codes, bounding the size of Kakeya sets, and constructing randomness extractors. Specifically, the method is used to improve bounds on list decoding, show that certain Kakeya sets must be large, and allow extraction of more randomness from weak sources. Propagating multiplicities of derivatives allows tighter analysis of these problems.
The document summarizes research on multiple-conclusion calculi for first-order Gödel logic. It introduces Gödel logic and describes its semantics using both many-valued semantics based on truth values in the interval [0,1] and Kripke-style semantics. It then outlines proof theory for Gödel logic, including early sequent calculi and more recent hypersequent calculi. The hypersequent calculus introduced in 1991 uses standard rules and has been extended to the first-order case. The document provides details on the structural and logical rules of this single-conclusion hypersequent system.
The document summarizes a talk on polynomial identity testing (PIT). PIT is the problem of determining if a polynomial computed by an arithmetic circuit is identical to the zero polynomial. The talk outlines the definition of PIT, its connection to circuit lower bounds, and surveys positive results for restricted circuit classes. It also provides examples of proof techniques for PIT on depth-3 and depth-4 circuits and discusses the relationship between PIT and polynomial factorization.
This document summarizes an algorithm for maximizing throughput in online scheduling of equal length jobs. The algorithm aims to schedule incoming jobs with the goal of maximizing total value of completed jobs by their deadlines. It uses a charging scheme and potential function to prove it is (2+√5)-competitive, an improvement over prior algorithms. The algorithm handles jobs arriving online with weights, processing times, deadlines, and considers models where preemption allows restarting or resuming previously completed work. Open questions remain around settling the exact competitive ratio and developing new algorithmic methods.
The document discusses efficient algorithms for performing approximate matching queries on strings that have been grammar-compressed. It introduces the concept of implicit unit-Monge matrices which can represent permutation matrices in a space-efficient way using a range tree data structure. This representation allows dominance counting queries, needed for string comparison, to be performed in O(log2 n) time after an O(n log n) preprocessing step. More advanced data structures can improve these asymptotic time and space bounds further.
This document presents an overview of the consensus problem from an informal and formal perspective. It discusses how consensus requires representativity, where the decision reflects a sufficient number of individual opinions, and stability, where the decision is robust to individual opinion variations. It also presents some key formalizations, including defining consensus as a function from the set of sensor inputs and memory states to decisions. It introduces the concept of a geodesic to measure stability as the maximum number of state transitions needed to return to the starting configuration along a trajectory where each sensor changes at most once.
The document presents a polynomial-time algorithm for finding a minimal conflicting set of rows (MCSR) in a binary matrix that contains a given row. It defines MCSR as a set of rows that does not have the consecutive ones property but where any proper subset does have the property. The algorithm works by representing the binary matrix as a vertex-colored bipartite graph and detecting forbidden substructures called Tucker configurations that characterize when the consecutive ones property does not hold. It finds an MCSR containing the given row by pruning rows from the graph until a Tucker configuration exists using the current set but not with any proper subset.
The document discusses locally decodable codes, which allow recovery of individual data symbols from a coded data set even after erasures. Reed-Muller codes and multiplicity codes were early constructions that provided locality but only up to a rate of 0.5. Matching vector codes were later introduced and can achieve locality r for codes of positive rate and length n=O(r^2). However, the optimal tradeoff between rate, length, and locality remains an open problem.
The document discusses locally decodable codes, which allow recovery of individual data symbols from a coded data set even after erasures. Reed-Muller codes and multiplicity codes were early constructions that provided locality but only up to a rate of 0.5. Matching vector codes were later introduced and can achieve locality r for codes of positive rate and length n=O(r^2). However, the optimal tradeoff between rate, length, and locality remains an open problem.
This document discusses the relationships between orbits of linear maps and regular languages. It shows that the chamber hitting problem (CHP) and permutation filter realizability problem are Turing equivalent. It also shows that the injective filter and surjective filter realizability problems are decidable by reducing them to problems about orbits. However, the regular realizability problem for the track product of the periodic and permutation filters is undecidable, as it can reduce the undecidable zero in the upper right corner problem.
The document summarizes precedence automata and languages. It provides historical background on operator precedence grammars and Floyd languages. It then discusses how precedence parsing works using an example arithmetic expression. Key points include using a precedence table to determine parentheses insertion and defining three types of moves for an automata model based on symbol precedence: push, mark, and flush. The example demonstrates the automata processing a Dyck language expression.
The document discusses the constraint satisfaction problem (CSP) and the dichotomy conjecture regarding the complexity of CSP instances. It provides definitions and examples of CSPs. It explains the role of polymorphisms in determining the complexity, identifying semilattice, majority and affine polymorphisms as "good". It outlines the dichotomy conjecture that CSPs are either solvable in polynomial time or NP-complete depending on the presence of certain types of local structure defined by polymorphisms. The document also discusses algorithms and results for various constraint languages.
This document describes a Synchronized Alternating Pushdown Automaton (SAPDA) that accepts the language of reduplication with a center marker (RCM). The SAPDA utilizes recursive conjunctive transitions to check that the nth letter before the center marker '$' is the same as the nth letter from the end of the string, for all letters n. This allows the SAPDA to accept strings of the form w$w, where w is any string over the alphabet {a,b}. The construction of the SAPDA involves states that check specific letters at specific positions relative to the center marker.
The document discusses the constraint satisfaction problem (CSP) and the dichotomy conjecture in computational complexity theory. It defines CSP and provides examples. It discusses the role of polymorphisms - operations that preserve constraints. The presence or absence of certain polymorphisms like semilattice, majority, and affine operations determines the complexity of CSP for a given constraint language. The document outlines a proposed dichotomy - CSP is either solvable in polynomial time or NP-complete, depending on the polymorphisms. It surveys partial results proving this conjecture and algorithms for certain constraint languages.
The document discusses shared-memory systems and charts. It provides definitions and concepts related to modeling shared-memory concurrency using partial orders of events called pomsets. Specifically, it defines:
- Shared-memory systems as consisting of registers, data, processes, actions, and rules for updating configurations.
- Pomsets as labeled partial orders used to model executions.
- The may-occur-concurrently relation for rules in a shared-memory system.
- Partial-order semantics for runs of pomsets in a shared-memory system.
- Shared-memory charts (SMCs) as pomsets with gates used to model specifications.
The document discusses precedence automata and languages. It provides historical background on operator precedence grammars and related families of languages. As an example, it explains how parsing an arithmetic expression like 4+5×6 works according to an implicit context-free grammar and by respecting the precedence of operators. It introduces the concept of a precedence table to determine the admissible parentheses generators between pairs of symbols in a grammar.
The document discusses computational models for algebraic decision trees and algebraic computation trees over a ground field F. It describes how algebraic decision trees use polynomials of degree ≤ d to branch at each node, while algebraic computation trees allow testing polynomials to be calculated from previous polynomials along the path. The document then covers existing lower bounds on the complexity C(S) of the membership problem for a set S in terms of topological invariants of S, such as the number of connected components, Euler characteristic, and sum of Betti numbers.
The document discusses recognizing sparse perfect elimination bipartite graphs. It begins with an example of Gaussian elimination on a matrix that introduces new non-zero values. The key points are that perfect elimination bipartite graphs correspond to matrices that can be eliminated without creating new non-zeros, and this can be achieved by finding a sequence of bisimplicial edges in the corresponding bipartite graph. The document proposes using bisimplicial edges as pivots during elimination to avoid introducing new non-zeros.
The document discusses recognizing sparse perfect elimination bipartite graphs through matrix elimination. It provides an example of Gaussian elimination on a matrix that introduces new non-zero values. The key points are:
- Perfect elimination bipartite graphs correspond to matrices that allow elimination without creating new non-zeros.
- Existing algorithms have time complexity of O(n^5) or O(n^3/log n) but may produce dense matrices from sparse ones.
- A new algorithm is proposed that avoids this issue by working directly with the sparse matrix structure.
The document discusses the method of multiplicities, which is a technique for combinatorics using algebra. It involves finding a polynomial that vanishes on a set with high multiplicity. This is applied to problems in list decoding of Reed-Solomon codes, bounding the size of Kakeya sets, and constructing randomness extractors. Specifically, the method is used to improve bounds on list decoding, show that certain Kakeya sets must be large, and allow extraction of more randomness from weak sources. Propagating multiplicities of derivatives allows tighter analysis of these problems.
The document summarizes research on multiple-conclusion calculi for first-order Gödel logic. It introduces Gödel logic and describes its semantics using both many-valued semantics based on truth values in the interval [0,1] and Kripke-style semantics. It then outlines proof theory for Gödel logic, including early sequent calculi and more recent hypersequent calculi. The hypersequent calculus introduced in 1991 uses standard rules and has been extended to the first-order case. The document provides details on the structural and logical rules of this single-conclusion hypersequent system.
The document summarizes a talk on polynomial identity testing (PIT). PIT is the problem of determining if a polynomial computed by an arithmetic circuit is identical to the zero polynomial. The talk outlines the definition of PIT, its connection to circuit lower bounds, and surveys positive results for restricted circuit classes. It also provides examples of proof techniques for PIT on depth-3 and depth-4 circuits and discusses the relationship between PIT and polynomial factorization.
This document summarizes an algorithm for maximizing throughput in online scheduling of equal length jobs. The algorithm aims to schedule incoming jobs with the goal of maximizing total value of completed jobs by their deadlines. It uses a charging scheme and potential function to prove it is (2+√5)-competitive, an improvement over prior algorithms. The algorithm handles jobs arriving online with weights, processing times, deadlines, and considers models where preemption allows restarting or resuming previously completed work. Open questions remain around settling the exact competitive ratio and developing new algorithmic methods.
The document discusses efficient algorithms for performing approximate matching queries on strings that have been grammar-compressed. It introduces the concept of implicit unit-Monge matrices which can represent permutation matrices in a space-efficient way using a range tree data structure. This representation allows dominance counting queries, needed for string comparison, to be performed in O(log2 n) time after an O(n log n) preprocessing step. More advanced data structures can improve these asymptotic time and space bounds further.
This document presents an overview of the consensus problem from an informal and formal perspective. It discusses how consensus requires representativity, where the decision reflects a sufficient number of individual opinions, and stability, where the decision is robust to individual opinion variations. It also presents some key formalizations, including defining consensus as a function from the set of sensor inputs and memory states to decisions. It introduces the concept of a geodesic to measure stability as the maximum number of state transitions needed to return to the starting configuration along a trajectory where each sensor changes at most once.
The document presents a polynomial-time algorithm for finding a minimal conflicting set of rows (MCSR) in a binary matrix that contains a given row. It defines MCSR as a set of rows that does not have the consecutive ones property but where any proper subset does have the property. The algorithm works by representing the binary matrix as a vertex-colored bipartite graph and detecting forbidden substructures called Tucker configurations that characterize when the consecutive ones property does not hold. It finds an MCSR containing the given row by pruning rows from the graph until a Tucker configuration exists using the current set but not with any proper subset.
The document discusses locally decodable codes, which allow recovery of individual data symbols from a coded data set even after erasures. Reed-Muller codes and multiplicity codes were early constructions that provided locality but only up to a rate of 0.5. Matching vector codes were later introduced and can achieve locality r for codes of positive rate and length n=O(r^2). However, the optimal tradeoff between rate, length, and locality remains an open problem.
The document discusses locally decodable codes, which allow recovery of individual data symbols from a coded data set even after erasures. Reed-Muller codes and multiplicity codes were early constructions that provided locality but only up to a rate of 0.5. Matching vector codes were later introduced and can achieve locality r for codes of positive rate and length n=O(r^2). However, the optimal tradeoff between rate, length, and locality remains an open problem.
This document discusses the relationships between orbits of linear maps and regular languages. It shows that the chamber hitting problem (CHP) and permutation filter realizability problem are Turing equivalent. It also shows that the injective filter and surjective filter realizability problems are decidable by reducing them to problems about orbits. However, the regular realizability problem for the track product of the periodic and permutation filters is undecidable, as it can reduce the undecidable zero in the upper right corner problem.
The document summarizes precedence automata and languages. It provides historical background on operator precedence grammars and Floyd languages. It then discusses how precedence parsing works using an example arithmetic expression. Key points include using a precedence table to determine parentheses insertion and defining three types of moves for an automata model based on symbol precedence: push, mark, and flush. The example demonstrates the automata processing a Dyck language expression.
The document discusses the constraint satisfaction problem (CSP) and the dichotomy conjecture regarding the complexity of CSP instances. It provides definitions and examples of CSPs. It explains the role of polymorphisms in determining the complexity, identifying semilattice, majority and affine polymorphisms as "good". It outlines the dichotomy conjecture that CSPs are either solvable in polynomial time or NP-complete depending on the presence of certain types of local structure defined by polymorphisms. The document also discusses algorithms and results for various constraint languages.
This document describes a Synchronized Alternating Pushdown Automaton (SAPDA) that accepts the language of reduplication with a center marker (RCM). The SAPDA utilizes recursive conjunctive transitions to check that the nth letter before the center marker '$' is the same as the nth letter from the end of the string, for all letters n. This allows the SAPDA to accept strings of the form w$w, where w is any string over the alphabet {a,b}. The construction of the SAPDA involves states that check specific letters at specific positions relative to the center marker.
The document discusses the constraint satisfaction problem (CSP) and the dichotomy conjecture in computational complexity theory. It defines CSP and provides examples. It discusses the role of polymorphisms - operations that preserve constraints. The presence or absence of certain polymorphisms like semilattice, majority, and affine operations determines the complexity of CSP for a given constraint language. The document outlines a proposed dichotomy - CSP is either solvable in polynomial time or NP-complete, depending on the polymorphisms. It surveys partial results proving this conjecture and algorithms for certain constraint languages.
The document discusses shared-memory systems and charts. It provides definitions and concepts related to modeling shared-memory concurrency using partial orders of events called pomsets. Specifically, it defines:
- Shared-memory systems as consisting of registers, data, processes, actions, and rules for updating configurations.
- Pomsets as labeled partial orders used to model executions.
- The may-occur-concurrently relation for rules in a shared-memory system.
- Partial-order semantics for runs of pomsets in a shared-memory system.
- Shared-memory charts (SMCs) as pomsets with gates used to model specifications.
The document discusses precedence automata and languages. It provides historical background on operator precedence grammars and related families of languages. As an example, it explains how parsing an arithmetic expression like 4+5×6 works according to an implicit context-free grammar and by respecting the precedence of operators. It introduces the concept of a precedence table to determine the admissible parentheses generators between pairs of symbols in a grammar.
1. Towards a Complexity Theory for
Randomized Search Heuristics:
The Ranking-Based Black-Box Model
Benjamin Doerr / Carola Winzen
CSR, June 14, 2011
2. Our Motivation
General-purpose (randomized) search heuristics are
...easy to implement,
...very flexible,
...need little a priori knowledge about problem at hand,
...can deal with many constraints and nonlinearities,
B. Doerr/C. Winzen: Ranking-Based Black-Box Model CSR, June 14, 2011
3. Our Motivation
General-purpose (randomized) search heuristics are
...easy to implement,
...very flexible,
...need little a priori knowledge about problem at hand,
...can deal with many constraints and nonlinearities,
and thus, frequently applied in practice.
Some theoretical results exist.
Typical result: runtime analysis for
a particular algorithm
on a particular problem.
B. Doerr/C. Winzen: Ranking-Based Black-Box Model CSR, June 14, 2011
4. Our Motivation
General-purpose (randomized) search heuristics are
...easy to implement,
...very flexible,
...need little a priori knowledge about problem at hand,
...can deal with many constraints and nonlinearities,
and thus, frequently applied in practice.
Some theoretical results exist.
Typical result: runtime analysis for Comparable to
a particular algorithm the “early days”
on a particular problem. of Computer
Science
B. Doerr/C. Winzen: Ranking-Based Black-Box Model CSR, June 14, 2011
5. Our Motivation
In classical theoretical computer science:
first results: runtime analysis for
a particular algorithm
on a particular problem.
B. Doerr/C. Winzen: Ranking-Based Black-Box Model CSR, June 14, 2011
6. Our Motivation
In classical theoretical computer science:
first results: runtime analysis for
a particular algorithm
on a particular problem.
general lower bounds (“tractability of a problem”)
complexity theory
B. Doerr/C. Winzen: Ranking-Based Black-Box Model CSR, June 14, 2011
7. Our Motivation
In classical theoretical computer science:
first results: runtime analysis for
a particular algorithm
on a particular problem.
general lower bounds (“tractability of a problem”)
complexity theory
How to create a
complexity theory for
randomized search
heuristics?
B. Doerr/C. Winzen: Ranking-Based Black-Box Model CSR, June 14, 2011
8. Our Motivation
In classical theoretical computer science:
first results: runtime analysis for
a particular algorithm
on a particular problem.
general lower bounds (“tractability of a problem”)
complexity theory
Our aim: to understand the tractability of a problem
for general-purpose (randomized) search heuristics
“Towards a Complexity Theory for
Randomized Search Heuristics”
B. Doerr/C. Winzen: Ranking-Based Black-Box Model CSR, June 14, 2011
9. A General View on Search Heuristics
Search
Heuristic f
Black-Box
= “Oracle”
B. Doerr/C. Winzen: Ranking-Based Black-Box Model CSR, June 14, 2011
10. A General View on Search Heuristics
x
Search
Heuristic f
Black-Box
= “Oracle”
B. Doerr/C. Winzen: Ranking-Based Black-Box Model CSR, June 14, 2011
11. A General View on Search Heuristics
x
Search
Heuristic f
f(x)
x
Black-Box
= “Oracle”
B. Doerr/C. Winzen: Ranking-Based Black-Box Model CSR, June 14, 2011
12. A General View on Search Heuristics
x
Search
Heuristic f
f(x)
x
Black-Box
= “Oracle”
Typical cost measure: number of function evaluations
until an optimal solution is queried for the first time
B. Doerr/C. Winzen: Ranking-Based Black-Box Model CSR, June 14, 2011
13. A General View on Search Heuristics
Classical Query Complexity Model
x
Search
Heuristic f
f(x)
x
Black-Box
= “Oracle”
Typical cost measure: number of function evaluations
until an optimal solution is queried for the first time
B. Doerr/C. Winzen: Ranking-Based Black-Box Model CSR, June 14, 2011
14. A Theory for (Randomized) Search Heuristics
Part 1: Classical query complexity model
Game theoretic view
Example: Mastermind
Part 2: Refinement: ranking-based query complexity
“Towards a Complexity Theory for
Randomized Search Heuristics:
The Ranking-Based Black-Box Model”
B. Doerr/C. Winzen: Ranking-Based Black-Box Model CSR, June 14, 2011
15. Example: A Mastermind Problem
Carole (=oracle) chooses a binary string of length n:
1 1 0 1 0 0 1 1
B. Doerr/C. Winzen: Ranking-Based Black-Box Model CSR, June 14, 2011
16. Example: A Mastermind Problem
Carole (=oracle) chooses a binary string of length n:
1 1 0 1 0 0 1 1
Paul (=algo.) tries to find it. He may ask any string of length n:
B. Doerr/C. Winzen: Ranking-Based Black-Box Model CSR, June 14, 2011
17. Example: A Mastermind Problem
Carole (=oracle) chooses a binary string of length n:
1 1 0 1 0 0 1 1
Paul (=algo.) tries to find it. He may ask any string of length n:
1 0 1 0 1 0 1 0
B. Doerr/C. Winzen: Ranking-Based Black-Box Model CSR, June 14, 2011
18. Example: A Mastermind Problem
Carole (=oracle) chooses a binary string of length n:
1 1 0 1 0 0 1 1
Paul (=algo.) tries to find it. He may ask any string of length n:
1 0 1 0 1 0 1 0
Carole computes in how many positions the strings coincide:
B. Doerr/C. Winzen: Ranking-Based Black-Box Model CSR, June 14, 2011
19. Example: A Mastermind Problem
Carole (=oracle) chooses a binary string of length n:
1 1 0 1 0 0 1 1
Paul (=algo.) tries to find it. He may ask any string of length n:
1 0 1 0 1 0 1 0
Carole computes in how many positions the strings coincide:
1 0 1 0 1 0 1 0
B. Doerr/C. Winzen: Ranking-Based Black-Box Model CSR, June 14, 2011
20. Example: A Mastermind Problem
Carole (=oracle) chooses a binary string of length n:
1 1 0 1 0 0 1 1
Paul (=algo.) tries to find it. He may ask any string of length n:
1 0 1 0 1 0 1 0
Carole computes in how many positions the strings coincide:
1 0 1 0 1 0 1 0
“Paul, our strings coincide in 3 bits”
B. Doerr/C. Winzen: Ranking-Based Black-Box Model CSR, June 14, 2011
21. Example: A Mastermind Problem
Carole (=oracle) chooses a binary string of length n:
1 1 0 1 0 0 1 1
Paul (=algo.) tries to find it. He may ask any string of length n:
1 0 1 0 1 0 1 0
Carole computes in how many positions the strings coincide:
1 0 1 0 1 0 1 0
We say that the
“Paul, our strings coincide in 3 bits” “fitness” of
Paul’s string is
3
B. Doerr/C. Winzen: Ranking-Based Black-Box Model CSR, June 14, 2011
22. Example: A Mastermind Problem
Carole chooses a binary string of length n:
1 1 0 1 0 0 1 1
Paul tries to find it. He may ask any string of length n:
1 0 1 0 1 0 1 0 3
B. Doerr/C. Winzen: Ranking-Based Black-Box Model CSR, June 14, 2011
23. Example: A Mastermind Problem
Carole chooses a binary string of length n:
1 1 0 1 0 0 1 1
Paul tries to find it. He may ask any string of length n:
1 0 1 0 1 0 1 0 3
1 0 1 1 0 1 0 1
B. Doerr/C. Winzen: Ranking-Based Black-Box Model CSR, June 14, 2011
24. Example: A Mastermind Problem
Carole chooses a binary string of length n:
1 1 0 1 0 0 1 1
Paul tries to find it. He may ask any string of length n:
1 0 1 0 1 0 1 0 3
1 0 1 1 0 1 0 1
B. Doerr/C. Winzen: Ranking-Based Black-Box Model CSR, June 14, 2011
25. Example: A Mastermind Problem
Carole chooses a binary string of length n:
1 1 0 1 0 0 1 1
Paul tries to find it. He may ask any string of length n:
1 0 1 0 1 0 1 0 3
1 0 1 1 0 1 0 1 4
...
B. Doerr/C. Winzen: Ranking-Based Black-Box Model CSR, June 14, 2011
26. Example: A Mastermind Problem
Carole chooses a binary string of length n:
1 1 0 1 0 0 1 1
Paul tries to find it. He may ask any string of length n:
1 0 1 0 1 0 1 0 3
1 0 1 1 0 1 0 1 4
...
How many queries does Paul need, on average,
until he has identified Carole’s string?
B. Doerr/C. Winzen: Ranking-Based Black-Box Model CSR, June 14, 2011
27. Reminder
Our aim: To understand tractability of a problem
for general-purpose (randomized) search heuristics
Measure: number of function evaluations until
an optimal solution is queried for the first time
Our main interest: good lower bounds
B. Doerr/C. Winzen: Ranking-Based Black-Box Model CSR, June 14, 2011
28. The Master Mind Problem: What Search Heuristics Do
Paul tries to find Carole’s binary string of length n:
1 1 0 1 0 0 1 1
B. Doerr/C. Winzen: Ranking-Based Black-Box Model CSR, June 14, 2011
29. The Master Mind Problem: What Search Heuristics Do
Paul tries to find Carole’s binary string of length n:
1 1 0 1 0 0 1 1
First query is arbitrary:
1 0 1 0 1 0 1 0
B. Doerr/C. Winzen: Ranking-Based Black-Box Model CSR, June 14, 2011
30. The Master Mind Problem: What Search Heuristics Do
Paul tries to find Carole’s binary string of length n:
1 1 0 1 0 0 1 1
First query is arbitrary:
1 0 1 0 1 0 1 0 3
B. Doerr/C. Winzen: Ranking-Based Black-Box Model CSR, June 14, 2011
31. The Master Mind Problem: What Search Heuristics Do
Paul tries to find Carole’s binary string of length n:
1 1 0 1 0 0 1 1
First query is arbitrary:
1 0 1 0 1 0 1 0 3
Then flip exactly one bit (chosen u.a.r.):
1 1 1 0 1 0 1 0
B. Doerr/C. Winzen: Ranking-Based Black-Box Model CSR, June 14, 2011
32. The Master Mind Problem: What Search Heuristics Do
Paul tries to find Carole’s binary string of length n:
1 1 0 1 0 0 1 1
First query is arbitrary:
1 0 1 0 1 0 1 0 3
Then flip exactly one bit (chosen u.a.r.):
1 1 1 0 1 0 1 0 4
B. Doerr/C. Winzen: Ranking-Based Black-Box Model CSR, June 14, 2011
33. The Master Mind Problem: What Search Heuristics Do
Paul tries to find Carole’s binary string of length n:
1 1 0 1 0 0 1 1
First query is arbitrary:
1 0 1 0 1 0 1 0 3
Then flip exactly one bit (chosen u.a.r.):
1 1 1 0 1 0 1 0 4
And it continues with the better of the two:
1 1 1 0 1 1 1 0
B. Doerr/C. Winzen: Ranking-Based Black-Box Model CSR, June 14, 2011
34. The Master Mind Problem: What Search Heuristics Do
Paul tries to find Carole’s binary string of length n:
1 1 0 1 0 0 1 1
First query is arbitrary:
1 0 1 0 1 0 1 0 3
Then flip exactly one bit (chosen u.a.r.):
1 1 1 0 1 0 1 0 4
Random Local Search algorithm:
Θ(n log n) Coupon Collector [Folklore]
B. Doerr/C. Winzen: Ranking-Based Black-Box Model CSR, June 14, 2011
35. The Master Mind Problem: What Search Heuristics Do
Paul tries to find Carole’s binary string of length n:
1 1 0 1 0 0 1 1
First query is arbitrary:
1 0 1 0 1 0 1 0
B. Doerr/C. Winzen: Ranking-Based Black-Box Model CSR, June 14, 2011
36. The Master Mind Problem: What Search Heuristics Do
Paul tries to find Carole’s binary string of length n:
1 1 0 1 0 0 1 1
First query is arbitrary:
1 0 1 0 1 0 1 0 3
Flip each bit with probability 1/n:
B. Doerr/C. Winzen: Ranking-Based Black-Box Model CSR, June 14, 2011
37. The Master Mind Problem: What Search Heuristics Do
Paul tries to find Carole’s binary string of length n:
1 1 0 1 0 0 1 1
First query is arbitrary:
1 0 1 0 1 0 1 0 3
Flip each bit with probability 1/n:
0 0 1 0 1 1 1 0 1
B. Doerr/C. Winzen: Ranking-Based Black-Box Model CSR, June 14, 2011
38. The Master Mind Problem: What Search Heuristics Do
Paul tries to find Carole’s binary string of length n:
1 1 0 1 0 0 1 1
First query is arbitrary:
1 0 1 0 1 0 1 0 3
Flip each bit with probability 1/n:
0 0 1 0 1 1 1 0 1
And it continues with the better of the two
B. Doerr/C. Winzen: Ranking-Based Black-Box Model CSR, June 14, 2011
39. The Master Mind Problem: What Search Heuristics Do
Paul tries to find Carole’s binary string of length n:
1 1 0 1 0 0 1 1
First query is arbitrary:
1 0 1 0 1 0 1 0 3
Flip each bit with probability 1/n:
0 0 1 0 1 1 1 0 1
(1+1) Evolutionary Algorithm:
Θ(n log n)[Droste/Jansen/Wegener 92]
[Mühlenbein
02]
B. Doerr/C. Winzen: Ranking-Based Black-Box Model CSR, June 14, 2011
40. The Master Mind Problem: Optimal Strategies (1/2)
Paul tries to find Carole’s binary string of length n:
1 0 0 1 0 0 1 1
B. Doerr/C. Winzen: Ranking-Based Black-Box Model CSR, June 14, 2011
41. The Master Mind Problem: Optimal Strategies (1/2)
Paul tries to find Carole’s binary string of length n:
1 0 0 1 0 0 1 1
He can go through the string bit by bit:
0 0 0 0 0 0 0 0 4
B. Doerr/C. Winzen: Ranking-Based Black-Box Model CSR, June 14, 2011
42. The Master Mind Problem: Optimal Strategies (1/2)
Paul tries to find Carole’s binary string of length n:
1 0 0 1 0 0 1 1
He can go through the string bit by bit:
0 0 0 0 0 0 0 0 4
1 0 0 0 0 0 0 0 5
B. Doerr/C. Winzen: Ranking-Based Black-Box Model CSR, June 14, 2011
43. The Master Mind Problem: Optimal Strategies (1/2)
Paul tries to find Carole’s binary string of length n:
1 0 0 1 0 0 1 1
He can go through the string bit by bit:
0 0 0 0 0 0 0 0 4
1 0 0 0 0 0 0 0 5
1 1 0 0 0 0 0 0 4
B. Doerr/C. Winzen: Ranking-Based Black-Box Model CSR, June 14, 2011
44. The Master Mind Problem: Optimal Strategies (1/2)
Paul tries to find Carole’s binary string of length n:
1 0 0 1 0 0 1 1
He can go through the string bit by bit:
0 0 0 0 0 0 0 0 4
1 0 0 0 0 0 0 0 5
1 1 0 0 0 0 0 0 4
...
1 0 0 1 0 0 1 1 8
B. Doerr/C. Winzen: Ranking-Based Black-Box Model CSR, June 14, 2011
45. The Master Mind Problem: Optimal Strategies (1/2)
Paul tries to find Carole’s binary string of length n:
1 0 0 1 0 0 1 1
He can go through the string bit by bit:
0 0 0 0 0 0 0 0 4
1 0 0 0 0 0 0 0 5
1 1 0 0 0 0 0 0 4 O(n)
...
1 0 0 1 0 0 1 1 8
B. Doerr/C. Winzen: Ranking-Based Black-Box Model CSR, June 14, 2011
46. The Master Mind Problem: Optimal Strategies (1/2)
Paul tries to find Carole’s binary string of length n:
1 0 0 1 0 0 1 1
He can go through the string bit by bit:
0 0 0 0 0 0 0 0 4
1 0 0 0 0 0 0 0 5
1 1 0 0 0 0 0 0 4 O(n)
...
Can we do
1 0 0 1 0 0 1 1 even8
better?
B. Doerr/C. Winzen: Ranking-Based Black-Box Model CSR, June 14, 2011
47. The Master Mind Problem: Optimal Strategies (2/2)
1 1 0 1 0 0 1 1
B. Doerr/C. Winzen: Ranking-Based Black-Box Model CSR, June 14, 2011
48. The Master Mind Problem: Optimal Strategies (2/2)
1 1 0 1 0 0 1 1
1 1 1 1 0 1 0 0 1 4
2 0 1 0 1 0 0 0 0 5
cn
0 0 0 1 1 1 1 1 4
log n
B. Doerr/C. Winzen: Ranking-Based Black-Box Model CSR, June 14, 2011
55. Intermediate Summary
Want to understand tractability of a problem for general-
purpose (randomized) search heuristics
Query complexity as such is not a sufficient measure:
Mastermind problem
1 1 0 1 0 0 1 1
Search Heuristics Query Complexity
n
Θ(n log n) Θ( log n )
B. Doerr/C. Winzen: Ranking-Based Black-Box Model CSR, June 14, 2011
56. The Ranking-Based Black-Box Model
Observation: many randomized search heuristics use fitness
values only to compare
B. Doerr/C. Winzen: Ranking-Based Black-Box Model CSR, June 14, 2011
57. The Ranking-Based Black-Box Model
Observation: many randomized search heuristics use fitness
values only to compare
RLS (1+1) EA ...
B. Doerr/C. Winzen: Ranking-Based Black-Box Model CSR, June 14, 2011
58. The Ranking-Based Black-Box Model
Observation: many randomized search heuristics use fitness
values only to compare
RLS (1+1) EA ...
B. Doerr/C. Winzen: Ranking-Based Black-Box Model CSR, June 14, 2011
59. The Ranking-Based Black-Box Model
Does not reveal absolute fitness values:
Algorithm
f
Black-Box
= “Oracle”
B. Doerr/C. Winzen: Ranking-Based Black-Box Model CSR, June 14, 2011
60. The Ranking-Based Black-Box Model
Does not reveal absolute fitness values:
x
Algorithm
f
Black-Box
= “Oracle”
B. Doerr/C. Winzen: Ranking-Based Black-Box Model CSR, June 14, 2011
61. The Ranking-Based Black-Box Model
Does not reveal absolute fitness values:
x
Algorithm
f
Rank 1 x
Rank 2 x Black-Box
Rank 2 x = “Oracle”
Rank 4 x
B. Doerr/C. Winzen: Ranking-Based Black-Box Model CSR, June 14, 2011
62. The Ranking-Based Black-Box Model
Equivalent formulation: Let g : R → R be a strictly monotone function
x
Algorithm f
g
g(f(x))=127 Rank 1 g(f(x))
fx
g(f(x))=125 Rank 2 Black-Box
g(f(x))=125 Rank 2
g(f(x))=27 Rank 4
= “Oracle”
B. Doerr/C. Winzen: Ranking-Based Black-Box Model CSR, June 14, 2011
63. Intermediate Summary
Want to understand tractability of a problem
for general-purpose (randomized) search heuristics
Query complexity as such is not a sufficient measure
(Many) Randomized search heuristics do selection based on
relative fitness values only, not on absolute values:
Ranking-Based Black-Box Model
Does it Mastermind problem
help? 1 1 0 1 0 0 1 1
B. Doerr/C. Winzen: Ranking-Based Black-Box Model CSR, June 14, 2011
64. The Ranking-Based BBC of Mastermind is Θ(n / log n)
n
Mastermind problem
1 1 0 1 0 0 1 1
Classical
Query Complexity
n
Θ( log n )
Search Heuristics
Θ(n log n)
Ranking-Based
Query Complexity
n
Θ( log n )
B. Doerr/C. Winzen: Ranking-Based Black-Box Model CSR, June 14, 2011
65. Example: BinaryValue //Weighted Mastermind
Carole chooses a binary string of length n and a permutation σ
1 1 0 1 0 0 1 1
24 22 21 23 25 28 26 27 σ=(4 2 1 3 5 8 6 7)
Paul wants to find it. He may ask any string of length n:
1 0 1 0 1 0 1 0
Carole computes the weighted fitness value:
1 0 1 0 1 0 1 0
“Paul, your string has a score of 336 (=24+28+26)”
B. Doerr/C. Winzen: Ranking-Based Black-Box Model CSR, June 14, 2011
66. The Query Complexity of BinaryValue is O(log n)
Paul can do a binary search (parallel for each i≤n):
1 1 0 1 0 0 1 1
24 22 21 23 25 28 26 27
1 1 1 1 1 1 1 1 24+22+23+26+27
1 1 1 1 0 0 0 0 24+22+23+25+28
1 1 0 0 1 1 0 0 24+22+21
1 0 1 0 1 0 1 0 24+28+26
1 1 0 1 0 0 1 1
B. Doerr/C. Winzen: Ranking-Based Black-Box Model CSR, June 14, 2011
67. Binary Search not Possible in Ranking-Based Model
Paul can do a binary search (parallel for each i≤n):
1 1 0 1 0 0 1 1
24 22 21 23 25 28 26 27
1 1 1 1 1 1 1 1 28
1 1 1 1 0 0 0 0 317
1 1 0 0 1 1 0 0 -29
1 0 1 0 1 0 1 0 29
? ? ? ? ? ? ? ?
B. Doerr/C. Winzen: Ranking-Based Black-Box Model CSR, June 14, 2011
68. Binary Search not Possible in Ranking-Based Model
Paul can do a binary search (parallel for each i≤n):
1 1 0 1 0 0 1 1
24 22 21 23 25 28 26 27
1 1 1 1 1 1 1 1 28
1 1 1 1 0 0 0 0 317
In fact,
1 1 0 RBBBC(BinaryValue)=Θ(n)
0 1 1 0 0 -29
1 0 1 0 1 0 1 0 29
? ? ? ? ? ? ? ?
B. Doerr/C. Winzen: Ranking-Based Black-Box Model CSR, June 14, 2011
69. The Ranking-Based Black-Box Complexity of BinaryValue is Θ(n)
n
Limited
If Algorithm queries two strings x and y,
Learning
it can learn at most 1 bit of the target string z.
t=2
Example:
g(BVz,σ (x)) > g(BVz,σ (y))
x=100000 ⇔
y=000000 z1 = x1 = 1
B. Doerr/C. Winzen: Ranking-Based Black-Box Model CSR, June 14, 2011
70. The Ranking-Based Black-Box Complexity of BinaryValue is Θ(n)
n
Limited
If Algorithm queries two strings x and y,
Learning
it can learn at most 1 bit of the target string z.
t=2
Limited
If Algorithm queries t strings x1,...,xt,
x
Learning
it can learn at most t-1 bit of the target string z.
general t
B. Doerr/C. Winzen: Ranking-Based Black-Box Model CSR, June 14, 2011
71. The Ranking-Based Black-Box Complexity of BinaryValue is Θ(n)
n
Limited
If Algorithm queries two strings x and y,
Learning
it can learn at most 1 bit of the target string z.
t=2
Limited
If Algorithm queries t strings x1,...,xt,
x
Learning
it can learn at most t-1 bit of the target string z.
general t
Ω(n)
There is no deterministic algorithm which
deterministic
optimizes BinaryValue in sublinear time.
lower bound
B. Doerr/C. Winzen: Ranking-Based Black-Box Model CSR, June 14, 2011
72. The Ranking-Based Black-Box Complexity of BinaryValue is Θ(n)
n
Limited
If Algorithm queries two strings x and y,
Learning
it can learn at most 1 bit of the target string z.
t=2
Limited
If Algorithm queries t strings x1,...,xt,
x
Learning
it can learn at most t-1 bit of the target string z.
general t
Ω(n)
There is no deterministic algorithm which
deterministic
optimizes BinaryValue in sublinear time.
lower bound
Ω(n)
Follows from deterministic lower bound and
randomized
Yao’s minimax principle
lower bound
B. Doerr/C. Winzen: Ranking-Based Black-Box Model CSR, June 14, 2011
73. Summary
Want to understand tractability of different problems
for (randomized) search heuristics
Measure: number of function evaluations
B. Doerr/C. Winzen: Ranking-Based Black-Box Model CSR, June 14, 2011
74. Summary
Want to understand tractability of different problems
for (randomized) search heuristics
Measure: number of function evaluations
Our main interest are good lower bounds
Classical query complexity: often too weak lower bounds
B. Doerr/C. Winzen: Ranking-Based Black-Box Model CSR, June 14, 2011
75. Summary
Want to understand tractability of different problems
for (randomized) search heuristics
Measure: number of function evaluations
Our main interest are good lower bounds
Classical query complexity: often too weak lower bounds
Ranking-Based Black-Box Model:
query complexity model
only relative, not absolute fitness values are given
B. Doerr/C. Winzen: Ranking-Based Black-Box Model CSR, June 14, 2011
76. Summary
Want to understand tractability of different problems
for (randomized) search heuristics
Measure: number of function evaluations
Our main interest are good lower bounds
Classical query complexity: often too weak lower bounds
Ranking-Based Black-Box Model:
only relative, not absolute fitness values are given
BinaryValue problem
11010011
Classical
Query Complexity
Θ(log n)
Search Heuristics
Θ(n log n)
Ranking-Based
Query Complexity
Θ(n)
B. Doerr/C. Winzen: Ranking-Based Black-Box Model CSR, June 14, 2011
77. Summary
Want to understand tractability of different problems
for (randomized) search heuristics
Measure: number of function evaluations
Our main interest are good lower bounds
Classical query complexity: often too weak lower bounds
Ranking-Based Black-Box Model:
only relative, not absolute fitness values are given
BinaryValue problem Mastermind problem
11010011 11010011
Classical Classical
Query Complexity Query Complexity
n
Θ(log n) Θ( log n )
Search Heuristics Search Heuristics
Θ(n log n) Θ(n log n)
Ranking-Based Ranking-Based
Query Complexity Query Complexity
n
Θ(n) Θ( log n )
B. Doerr/C. Winzen: Ranking-Based Black-Box Model CSR, June 14, 2011
78. Future Work
Plenty!
B. Doerr/C. Winzen: Ranking-Based Black-Box Model CSR, June 14, 2011
79. Future Work
Plenty!
Other black-box models:
restricted memory models
unbiased sampling strategies
combinations thereof
...
B. Doerr/C. Winzen: Ranking-Based Black-Box Model CSR, June 14, 2011
80. Future Work
Plenty!
Other black-box models:
restricted memory models
unbiased sampling strategies
combinations thereof
...
Combinatorial problems:
MST, SSSP problems
partition problem
...
...
B. Doerr/C. Winzen: Ranking-Based Black-Box Model CSR, June 14, 2011
81. Future Work
Plenty!
Other black-box models:
restricted memory models
unbiased sampling strategies
Almost equivalent problem:
combinations thereof n distinguishable balls of unknown weight
...
10
Combinatorial problems: 8 9
MST, SSSP problems 5 6 7
1 2 3 4
partition problem
... How often do you need to use the
balance to find a perfect partition
... of the balls?
B. Doerr/C. Winzen: Ranking-Based Black-Box Model CSR, June 14, 2011