Successfully reported this slideshow.                             Upcoming SlideShare
×

# Benchmarking parity games

1,193 views

Published on

Parity games are a formalism that is used in software verification. The question whether a computer system satisfies a given property can be described as a parity game.

The complexity of solving parity games (the problem is in NP and co-NP), makes that het problem is not only interesting from a software verification point of view, but also from a complexity theoretic perspective. As a consequence, in recent years a large number of algorithms and heuristics for solving parity games have been developed. However, in this papers often only the theoretical complexity of the algorithm is considered, or, if experiments are performed they are based on a small set of parity games.

In this talk, I introduce a large set of parity games that should lead to better and more complete comparisons of algorithms that have been introduced in the literature.

This talk was presented at FSEN 2015. See my blog at http://www.jeroenkeiren.nl/blog/paper-benchmarking-parity-games/ for a more detailed description, and a download of the corresponding paper.

Published in: Science
• Full Name
Comment goes here.

Are you sure you want to Yes No • Be the first to comment

• Be the first to like this

### Benchmarking parity games

1. 1. Benchmarking Parity Games FSEN 2014 Jeroen Keiren Open University of the Netherlands & VU University Amsterdam 24 April 2015 1 / 19
2. 2. Where are parity games used? Model Checking Equivalence Checking Satisﬁability/Validity of modal logic Synthesis 2 / 19
3. 3. Parity Games V A set of vertices. → ⊆ V × V An edge relation. Ω : V → N A priority mapping. , Two players (even, odd). (V , V ) A partition of V . 0 s 0 t 0 u 0 w 1 v =even =odd 3 / 19
4. 4. Parity Games V A set of vertices. → ⊆ V × V An edge relation. Ω : V → N A priority mapping. , Two players (even, odd). (V , V ) A partition of V . 0 s 0 t 0 u 0 w 1 v =even =odd 3 / 19
5. 5. Parity Games V A set of vertices. → ⊆ V × V An edge relation. Ω : V → N A priority mapping. , Two players (even, odd). (V , V ) A partition of V . 0 s 0 t 0 u 0 w 1 v =even =odd 3 / 19
6. 6. Parity Games V A set of vertices. → ⊆ V × V An edge relation. Ω : V → N A priority mapping. , Two players (even, odd). (V , V ) A partition of V . 0 s 0 t 0 u 0 w 1 v =even =odd 3 / 19
7. 7. Parity Games V A set of vertices. → ⊆ V × V An edge relation. Ω : V → N A priority mapping. , Two players (even, odd). (V , V ) A partition of V . 0 s 0 t 0 u 0 w 1 v =even =odd 3 / 19
8. 8. Parity Games V A set of vertices. → ⊆ V × V An edge relation. Ω : V → N A priority mapping. , Two players (even, odd). (V , V ) A partition of V . 0 s 0 t 0 u 0 w 1 v =even =odd 3 / 19
9. 9. Parity Games V A set of vertices. → ⊆ V × V An edge relation. Ω : V → N A priority mapping. , Two players (even, odd). (V , V ) A partition of V . 0 s 0 t 0 u 0 w 1 v =even =odd Winner? 3 / 19
10. 10. Parity Games V A set of vertices. → ⊆ V × V An edge relation. Ω : V → N A priority mapping. , Two players (even, odd). (V , V ) A partition of V . 0 s 0 t 0 u 0 w 1 v =even =odd Winner? Optimal strategies? 3 / 19
11. 11. Parity Games V A set of vertices. → ⊆ V × V An edge relation. Ω : V → N A priority mapping. , Two players (even, odd). (V , V ) A partition of V . 0 s 0 t 0 u 0 w 1 v =even =odd Winner? Optimal strategies? 3 / 19
12. 12. Parity Games V A set of vertices. → ⊆ V × V An edge relation. Ω : V → N A priority mapping. , Two players (even, odd). (V , V ) A partition of V . 0 s 0 t 0 u 0 w 1 v =even =odd Winner? Optimal strategies? 3 / 19
13. 13. Winning Parity Games Memoryless determinacy Partition (W , W ) of V Player has memoryless winning strategy from W , for ∈ { , } 4 / 19
14. 14. Solving Parity Games Solving a parity game: Determine partition (W , W ) Complexity: Problem is in NP ∩ co-NP Is it in P? 5 / 19
15. 15. Solving Parity Games Solving a parity game: Determine partition (W , W ) Complexity: Problem is in NP ∩ co-NP Is it in P? Open! 5 / 19
16. 16. Why benchmark? Complexity + applications ⇒ active research Algorithms for: solving simplifying reducing parity games How to compare new algorithms to existing ones? 6 / 19
17. 17. Existing practice Only theoretical analysis (big-O) Class of games that meets upper bound Random games (Very) small set of games Results from diﬀerent papers not comparable 7 / 19
18. 18. Requirements on Benchmarks Cover broad range of games: Diﬀerent problems Diﬀerent structural properties Games from the literature 8 / 19
19. 19. Contributions Set of parity games List of structural properties Analysis of games w.r.t. these properties 9 / 19
20. 20. Set of parity games Model checking: Communication protocols (C)ABP, BRP, SWP Cache coherence protocol Two-player board games Industrial IEEE-1394 link-layer, truck lift Elevator, Hanoi towers Equivalence checking: strong-, weak-, branching bisimulation of communication protocols Validity/satisﬁability of LTL, CTL, CTL*, PDL and µ-calculus (using MLSolver) Random games (using PGSolver) Hard cases (using PGSolver) 10 / 19
21. 21. Structural properties Some properties known to aﬀect complexity of solving: Number of vertices and edges (“size”) Number of priorities Width measures (tree-width, DAG-width, etc.) SCCs New: alternation depth (inspired by modal equation systems) And some more. . . 11 / 19
22. 22. Alternation depth Describe complexity more accurately Similar to ideas in [Emerson & Lee 1986] for µ-calculus Three steps (let C ∈ sccs(G)) 1. Nesting depth of v in C is #alternations between even and odd priorities on paths of descending priorities in C 2. Nesting depth of C is max{nestingdepth(v) | v ∈ C} 3. Alternation depth of of a parity game is the maximal nesting depth of its SCCs 12 / 19
23. 23. Analysis of games w.r.t. structural properties Vertices vs. edges 101 102 103 104 105 106 107 101 102 103 104 105 106 107 108 Vertices Edges modelchecking equivalence mlsolver specialcases random 13 / 19
24. 24. Analysis of games w.r.t. structural properties Alternation depth 101 102 103 104 105 106 107 100 101 102 103 104 Vertices Alternationdepth modelchecking equivalence mlsolver specialcases random 14 / 19
25. 25. Analysis of games w.r.t. structural properties Diameter 101 102 103 104 100 101 102 103 104 BFS Height Diameter modelchecking equivalence mlsolver specialcases random 15 / 19
26. 26. Applications Used to assess parity game reductions in [Cranen, K & Willemse 2011,2012] Subset of generation process used for benchmarks in [K, Wesselink & Willemse, 2014] Conﬁrmed observation from [Friedmann & Lange 2009]: recursive algorithm beats sophisticated algorithms (unpublished) 16 / 19
27. 27. Summary I presented: A set of parity games Structural properties of parity games An analysis of the games w.r.t. these properties 17 / 19
28. 28. Open issues Use structural properties to optimise/design algorithms Perform large-scale comparison of diﬀerent algorithms Extend set of games with other encodings/more examples Design algorithms for computing more complex structural properties 18 / 19
29. 29. Please contribute your own games! jeroenkeiren.nl github.com/jkeiren/paritygame-generator 19 / 19