Successfully reported this slideshow.
Upcoming SlideShare
×

# MapReduce for Parallel Trace Validation of LTL Properties

1,214 views

Published on

We present an algorithm for the automated verification of Linear Temporal Logic formulae on event traces using an increasingly popular cloud computing framework called MapReduce. The algorithm can process multiple, arbitrary fragments of the trace in parallel, and compute its final result through a cycle of runs of MapReduce instances. Compared to classical, single-instance solutions, a proof-of-concept implementation shows through experimental evaluation how the algorithm reduces by as much as 90% the number of operations that must be performed linearly, resulting in a commensurate speed gain.

• Full Name
Comment goes here.

Are you sure you want to Yes No
• Be the first to comment

• Be the first to like this

### MapReduce for Parallel Trace Validation of LTL Properties

1. 1. MapReduce for ParallelTrace Validation of LTL PropertiesBenjamin Barre, Mathieu Klein, Maxime Soucy-Boivin, Pierre-Antoine Ollivier and Sylvain Hallé Université du Québec à Chicoutimi CANADA Fonds de recherche Nature et NSERC technologies CRSNG
2. 2. System
3. 3. System
4. 4. Instrumentation System
5. 5. Instrumentation System
6. 6. Instrumentation Trace System
7. 7. Instrumentation Trace Events System
8. 8. Instrumentation Trace Events System
9. 9. Trace validationInstrumentation Trace Events System
10. 10. or<T>Iterat
11. 11. hasNext or<T> Iterat next
12. 12. A call to next must be preceded by a call to hasNexthasNext or<T> Iterat next
13. 13. AB
14. 14. No CartCreate request can occurbefore a LoginResponse message A B
22. 22. Let A be a set of event symbols.A trace m is a mapping from ℕ tothe set of events :ℕ 0 1 2 3 4 ...A a a b c b
23. 23. X next ¬ ∧ → G globally A + ∧¬→ + F eventually U untilGround Boolean Temporal terms connectives operators = Linear Temporal Logic
24. 24. Let Φ be the set of all possible LTL formulas.The function ℒ : Φ → 2ℕ labels each state witha set of LTL formulasℕ 0 1 2 3 4 ...A a a b c b
25. 25. Let Φ be the set of all possible LTL formulas.The function ℒ : Φ → 2ℕ labels each state witha set of LTL formulasℒ b) ∨c b) b b b (a→ (a→ a∧c b a∧ a∧ b∨ G Gℕ 0 1 2 3 4 ...A a a b c b Example: ℒ(a∧b) = {0,1,4,...}
26. 26. i ∈ ℒ(a) ⇔ m(i) = ai ∈ ℒ(¬φ) ⇔ i ∉ ℒ(φ)i ∈ ℒ(φ∧ψ) ⇔ i ∈ ℒ(φ) and i ∈ ℒ(ψ)i ∈ ℒ(φ∨ψ) ⇔ i ∈ ℒ(φ) or i ∈ ℒ(ψ)i ∈ ℒ(X φ) ⇔ i+1 ∈ ℒ(φ)i ∈ ℒ(G φ) ⇔ j ∈ ℒ(φ) for all j ≥ ii ∈ ℒ(F φ) ⇔ j ∈ ℒ(φ) for some j ≥ ii ∈ ℒ(φ U ψ) ⇔ j ∈ ℒ(ψ) for some j ≥ i and k ∈ ℒ(φ) for all j ≥ k ≥ i
27. 27. Theorem i ∈ ℒ(φ) exactly when the trace m(i), m(i+1), ... satisfies φ σ φ ψ 0 1 2 3 4 ...
28. 28. Theorem i ∈ ℒ(φ) exactly when the trace m(i), m(i+1), ... satisfies φ σ φ ψ 0 1 2 3 4 ... Therefore... 0 ∈ ℒ(φ) ⇔ m ⊧ φ
29. 29. A call to next must be followed by a callto hasNextNo CartCreate request can occurbefore a LoginResponse messageA received order must eventuallybe shippedThree successive login attempts shouldtrigger an alarm
30. 30. A call to next must be followed by a callto hasNext G (next → X hasNext)No CartCreate request can occurbefore a LoginResponse messageA received order must eventuallybe shippedThree successive login attempts shouldtrigger an alarm
31. 31. A call to next must be followed by a callto hasNext G (next → X hasNext)No CartCreate request can occurbefore a LoginResponse message ¬ CartCreate U hasNextA received order must eventuallybe shippedThree successive login attempts shouldtrigger an alarm
32. 32. A call to next must be followed by a callto hasNext G (next → X hasNext)No CartCreate request can occurbefore a LoginResponse message ¬ CartCreate U hasNextA received order must eventuallybe shipped G (receive → F ship)Three successive login attempts shouldtrigger an alarm
33. 33. A call to next must be followed by a callto hasNext G (next → X hasNext)No CartCreate request can occurbefore a LoginResponse message ¬ CartCreate U hasNextA received order must eventuallybe shipped G (receive → F ship)Three successive login attempts shouldtrigger an alarm G ¬(fail ∧ (X (fail ∧ X fail)))
34. 34. Iterat or<T> Java MOP
35. 35. � The trace must be read linearly 2 3 4 5 1 � The algorithm works on a x1 single process / core / site
36. 36. 10,000,000 1,000,000 100,000 10,000 1,000 100 10 1 1970 1980 1990 2000 2010
37. 37. 10,000,000 Transistors (x1000) 1,000,000 100,000 10,000 1,000 100 10 1 1970 1980 1990 2000 2010
38. 38. 10,000,000 Transistors (x1000) 1,000,000 100,000 10,000 CPU Speed 1,000 (MHz) 100 10 1 1970 1980 1990 2000 2010
39. 39. PageRank f ∞
40. 40. ValueKey a 1{ Tuple (baaah)
41. 41. Data source
43. 43. Data sourceII a 2 z 7 . . .Input reader
44. 44. . . . a 2 a 2
45. 45. Mapper. . . a 2 a 2 M
46. 46. Mapper. . . a 2 a 2 M w 6 a 2 . . .
47. 47. Mapper. . . a 2 a 2 M w 6 a 2 . . . a b3 a2 3 3 g b
48. 48. a 3 a3 2a e 38 a a3 b 9 bb a 3 ba a
49. 49. e 38 a b . . . . . . b db Shuffling aa ba ab a a 2 a b 9 3
50. 50. b 9 ba a 2 2a 3 a
51. 51. b 9 ba a 2 2a 3 a Rb Ra Reducer
52. 52. b 9 ba a 2 2a 3 a Rb e 7 i 0 Ra z 8 x 2 . . . Reducer
53. 53. a b a a b a
54. 54. a b a a b aa aa bb a
55. 55. a b a a b aa a Ia bb a
56. 56. a b a a b a 〈a,1〉a a I 〈a,1〉a bb a
57. 57. a b a a b a 〈a,1〉a a I 〈a,1〉 〈a,1〉a b I 〈b,1〉 〈b,1〉b a I 〈a,1〉
58. 58. a b a a b a 〈a,1〉a a I 〈a,1〉 〈a,1〉a b I 〈b,1〉 〈b,1〉b a I 〈a,1〉
59. 59. a b a a b a 〈a,1〉a a I 〈a,1〉 〈a,1〉a b I 〈b,1〉 〈b,1〉b a I 〈a,1〉
60. 60. a b a a b a 〈a,1〉a a I 〈a,1〉 Ra 〈a,1〉a b I 〈b,1〉 〈b,1〉b a I 〈a,1〉
61. 61. a b a a b a 〈a,1〉a a I 〈a,1〉 Ra 〈a,4〉 〈a,1〉a b I 〈b,1〉 〈b,1〉b a I 〈a,1〉
62. 62. a b a a b a 〈a,1〉a a I 〈a,1〉 Ra 〈a,4〉 〈a,1〉a b I 〈b,1〉 〈b,1〉 Rb 〈b,2〉b a I 〈a,1〉
63. 63. a b a a b a 〈a,1〉a a I 〈a,1〉 Ra 〈a,4〉 〈a,1〉a b I 〈b,1〉 〈b,1〉 Rb 〈b,2〉b a I 〈a,1〉
64. 64. Superformula SuperformulaFormula G ∧ Subformula Subformula Subformula
65. 65. 4 GHeight 3 → 2 ∧ 3 1 2 ¬ F 1 0 0 a c b
66. 66. 4 GHeight 3 → 2 ∧ 3 1 2 ¬ F 1 0 0 a c b¬c has height 1G ((a ∧¬c) → F b) has height 4
67. 67. i ∈ ℒ(a) ⇔ m(i) = ai ∈ ℒ(¬φ) ⇔ i ∉ ℒ(φ)i ∈ ℒ(φ∧ψ) ⇔ i ∈ ℒ(φ) and i ∈ ℒ(ψ)i ∈ ℒ(φ∨ψ) ⇔ i ∈ ℒ(φ) or i ∈ ℒ(ψ)i ∈ ℒ(X φ) ⇔ i+1 ∈ ℒ(φ)i ∈ ℒ(G φ) ⇔ j ∈ ℒ(φ) for all j ≥ ii ∈ ℒ(F φ) ⇔ j ∈ ℒ(φ) for some j ≥ ii ∈ ℒ(φ U ψ) ⇔ j ∈ ℒ(ψ) for some j ≥ i and k ∈ ℒ(φ) for all j ≥ k ≥ i
68. 68. i ∈ ℒ(a) ⇔ m(i) = a i ∈ ℒ(¬φ) ⇔ i ∉ ℒ(φ) i ∈ ℒ(φ∧ψ) ⇔ i ∈ ℒ(φ) and i ∈ ℒ(ψ) i ∈ ℒ(φ∨ψ) ⇔ i ∈ ℒ(φ) or i ∈ ℒ(ψ) i ∈ ℒ(X φ) ⇔ i+1 ∈ ℒ(φ) i ∈ ℒ(G φ) ⇔ j ∈ ℒ(φ) for all j ≥ i i ∈ ℒ(F φ) ⇔ j ∈ ℒ(φ) for some j ≥ i i ∈ ℒ(φ U ψ) ⇔ j ∈ ℒ(ψ) for some j ≥ i and k ∈ ℒ(φ) for all j ≥ k ≥ iThe labelling of a formula depends onlyon labellings of formulas of strictly lower height
69. 69. i ∈ ℒ(a) ⇔ m(i) = a i ∈ ℒ(¬φ) ⇔ i ∉ ℒ(φ) i ∈ ℒ(φ∧ψ) ⇔ i ∈ ℒ(φ) and i ∈ ℒ(ψ) i ∈ ℒ(φ∨ψ) ⇔ i ∈ ℒ(φ) or i ∈ ℒ(ψ) i ∈ ℒ(X φ) ⇔ i+1 ∈ ℒ(φ) i ∈ ℒ(G φ) ⇔ j ∈ ℒ(φ) for all j ≥ i i ∈ ℒ(F φ) ⇔ j ∈ ℒ(φ) for some j ≥ i i ∈ ℒ(φ U ψ) ⇔ j ∈ ℒ(ψ) for some j ≥ i and k ∈ ℒ(φ) for all j ≥ k ≥ iThe labelling of a formula depends onlyon labellings of formulas of strictly lower height⇒ All labellings of formulas of same height are independent
70. 70. i ∈ ℒ(a) ⇔ m(i) = a i ∈ ℒ(¬φ) ⇔ i ∉ ℒ(φ) i ∈ ℒ(φ∧ψ) ⇔ i ∈ ℒ(φ) and i ∈ ℒ(ψ) i ∈ ℒ(φ∨ψ) ⇔ i ∈ ℒ(φ) or i ∈ ℒ(ψ) i ∈ ℒ(X φ) ⇔ i+1 ∈ ℒ(φ) i ∈ ℒ(G φ) ⇔ j ∈ ℒ(φ) for all j ≥ i i ∈ ℒ(F φ) ⇔ j ∈ ℒ(φ) for some j ≥ i i ∈ ℒ(φ U ψ) ⇔ j ∈ ℒ(ψ) for some j ≥ i and k ∈ ℒ(φ) for all j ≥ k ≥ iThe labelling of a formula depends onlyon labellings of formulas of strictly lower height⇒ All labellings of formulas of same height are independent ⇒ They can be computed in parallel
71. 71. M
72. 72. M Input: tuples 〈φ,(n,i)〉
73. 73. M Input: tuples 〈φ,(n,i)〉 “ n ∈ ℒ(φ), and the last cycle has evaluated labellings for formulas of height i ”
74. 74. “Lift” ℒ(φ) to superformulas of φM Input: tuples 〈φ,(n,i)〉 “ n ∈ ℒ(φ), and the last cycle has evaluated labellings for formulas of height i ”
75. 75. Output: tuples 〈ψ,(φ,n,i)〉 “Lift” ℒ(φ) to superformulas of φM Input: tuples 〈φ,(n,i)〉 “ n ∈ ℒ(φ), and the last cycle has evaluated labellings for formulas of height i ”
76. 76. Output: tuples 〈ψ,(φ,n,i)〉 “ n ∈ ℒ(φ), the last cycle has evaluated labellings for formulas of height i, and φ is a subformula of ψ ” “Lift” ℒ(φ) to superformulas of φM Input: tuples 〈φ,(n,i)〉 “ n ∈ ℒ(φ), and the last cycle has evaluated labellings for formulas of height i ”
77. 77.
78. 78. Input:〈ψ,(φ,n,i)〉 Rψ
79. 79. Input:〈ψ,(φ,n,i)〉 Rψ“ n ∈ ℒ(φ), the last cyclehas evaluated labellings forformulas of height i, andφ is a subformula of ψ ”
80. 80. Input: Compute ℒ(ψ)〈ψ,(φ,n,i)〉 Rψ“ n ∈ ℒ(φ), the last cyclehas evaluated labellings forformulas of height i, andφ is a subformula of ψ ”
81. 81. Input: Compute ℒ(ψ) Output:〈ψ,(φ,n,i)〉 〈ψ,(n,i+1)〉 Rψ“ n ∈ ℒ(φ), the last cyclehas evaluated labellings forformulas of height i, andφ is a subformula of ψ ”
82. 82. Input: Compute ℒ(ψ) Output:〈ψ,(φ,n,i)〉 〈ψ,(n,i+1)〉 Rψ“ n ∈ ℒ(φ), the last cycle “ n ∈ ℒ(ψ), and the lasthas evaluated labellings for cycle has evaluatedformulas of height i, and labellings for formulas ofφ is a subformula of ψ ” height i+1
83. 83. I
84. 84. Input: events (a,n) I
85. 85. Input: events (a,n) Output: tuples 〈ψ,(a,n,0)〉 I . . . “ n ∈ ℒ(a), the last cycle has evaluated labellings for formulas of height 0, and a is a subformula of ψ ”
86. 86. W
87. 87. Input: 〈ψ,(n,i)〉 W
88. 88. Output:Input: 〈ψ,(n,i)〉 True if 〈ψ,(0,i)〉 is read W False otherwise
89. 89. 3 . . . R W R 2 R R R 1 I . . . RI R R
90. 90. 3 . . . R W R 2 R R R 1 I . . . R I R RInputReaders generate the first tuples fromthe trace chunks
91. 91. 3 . . . R W R 2 R R R 1 I . . . R I R RThe tuples are shuffled to reducers that compute thelabelling ℒ for formulas of height 1
92. 92. 3 . . . R W R 2 R R R 1 I . . . R I R RMappers copy the labellings into tuples marked bysuperformulas of height 2
93. 93. 3 . . . R W R 2 R R R 1 I . . . R I R REach reducer computes the labelling of a formula ofheight 2 from the labelling of its subformulas
94. 94. 3 . . . R W R 2 R R R 1 I . . . R I R RMappers copy the labellings into tuples marked bysuperformulas of height 3
95. 95. 3 . . . R W R 2 R R R 1 I . . . R I R REach reducer computes the labelling of a formula ofheight 3 from the labelling of its subformulas
96. 96. 3 . . . R W R 2 R R R 1 I . . . R I R RAn output writer collects the resulting tuples, andoutputs “true” if it encounters a tuple for state 0
97. 97. a a b c b a ? G (¬a → F b) ⊨
98. 98. a a b c b a ? G (¬a → F b) ⊨(a,0) (b,2)(a,1) (c,3)(a,5) (b,4) 0 HEIGHT
99. 99. a a b c b a ? G (¬a → F b) ⊨(a,0) (b,2) I(a,1) (c,3) I(a,5) (b,4) I 0 HEIGHT
100. 100. a a b c b a ? G (¬a → F b) ⊨ 〈¬a,(a,0)〉(a,0) (b,2) I 〈F b,(b,2)〉(a,1) (c,3) I 〈¬a,(a,1)〉 〈F b,(b,4)〉(a,5) (b,4) I 0 1 〈¬a,(a,5)〉 HEIGHT
101. 101. a a b c b a ? G (¬a → F b) ⊨ 〈¬a,(a,0)〉(a,0) (b,2) I 〈F b,(b,2)〉 R ¬a(a,1) (c,3) I 〈¬a,(a,1)〉 R 〈F b,(b,4)〉 Fb(a,5) (b,4) I 0 1 〈¬a,(a,5)〉 HEIGHT
102. 102. a a b c b a ? G (¬a → F b) ⊨ 〈¬a,(a,0)〉 〈¬a,2〉(a,0) (b,2) I 〈¬a,3〉 〈F b,(b,2)〉 R ¬a 〈¬a,4〉(a,1) (c,3) I 〈¬a,(a,1)〉 〈F b,0〉 〈F b,1〉 R 〈F b,2〉 〈F b,(b,4)〉 Fb 〈F b,3〉(a,5) (b,4) I 0 1 〈¬a,(a,5)〉 b,4〉 〈FHEIGHT
103. 103. a a b c b a ? G (¬a → F b) ⊨〈¬a,2〉〈¬a,3〉 M〈¬a,4〉〈F b,0〉〈F b,1〉 M〈F b,2〉〈F b,3〉〈F b,4〉 M 2 HEIGHT
104. 104. a a b c b a ? G (¬a → F b) ⊨〈¬a,2〉 〈¬a → F b,(¬a,2)〉〈¬a,3〉 M 〈¬a → F b,(¬a,3)〉〈¬a,4〉 〈¬a → F b,(¬a,4)〉〈F b,0〉 〈¬a → F b,(F b,0)〉〈F b,1〉 〈¬a → F b,(F b,1)〉 M 〈¬a → F b,(F b,2)〉〈F b,2〉 〈¬a → F b,(F b,3)〉〈F b,3〉〈F b,4〉 M 〈¬a → F b,(F b,4)〉 2 HEIGHT
105. 105. a a b c b a ? G (¬a → F b) ⊨〈¬a,2〉 〈¬a → F b,(¬a,2)〉〈¬a,3〉 M 〈¬a → F b,(¬a,3)〉〈¬a,4〉 〈¬a → F b,(¬a,4)〉〈F b,0〉 〈¬a → F b,(F b,0)〉〈F b,1〉 〈¬a → F b,(F b,1)〉 R M 〈¬a → F b,(F b,2)〉 ¬a → Fb〈F b,2〉 〈¬a → F b,(F b,3)〉〈F b,3〉〈F b,4〉 M 〈¬a → F b,(F b,4)〉 2 HEIGHT
106. 106. a a b c b a ? G (¬a → F b) ⊨ 〈¬a → F b,0〉 〈¬a → F b,1〉 〈¬a → F b,2〉〈¬a,2〉 〈¬a → F b,(¬a,2)〉 〈¬a → F b,3〉〈¬a,3〉 M 〈¬a → F b,(¬a,3)〉 〈¬a → F b,4〉〈¬a,4〉 〈¬a → F b,(¬a,4)〉 〈¬a → F b,5〉〈F b,0〉 〈¬a → F b,(F b,0)〉〈F b,1〉 〈¬a → F b,(F b,1)〉 R M 〈¬a → F b,(F b,2)〉 ¬a → Fb〈F b,2〉 〈¬a → F b,(F b,3)〉〈F b,3〉〈F b,4〉 M 〈¬a → F b,(F b,4)〉 2 HEIGHT
107. 107. a a b c b a ? G (¬a → F b) ⊨〈¬a → F b,0〉〈¬a → F b,1〉 M〈¬a → F b,2〉〈¬a → F b,3〉 M〈¬a → F b,4〉〈¬a → F b,5〉 M 3 HEIGHT
108. 108. a a b c b a ? G (¬a → F b) ⊨ 〈G (¬a → F b),〈¬a → F b,0〉 (¬a → F b,0)〉〈¬a → F b,1〉 M 〈G (¬a → F b), (¬a → F b,1)〉 〈G (¬a → F b),〈¬a → F b,2〉 (¬a → F b,2)〉〈¬a → F b,3〉 M 〈G (¬a → F b), (¬a → F b,3)〉 〈G (¬a → F b),〈¬a → F b,4〉 (¬a → F b,4)〉〈¬a → F b,5〉 M 〈G (¬a → F b), 3 HEIGHT (¬a → F b,5)〉
109. 109. a a b c b a ? G (¬a → F b) ⊨ 〈G (¬a → F b),〈¬a → F b,0〉 (¬a → F b,0)〉〈¬a → F b,1〉 M 〈G (¬a → F b), (¬a → F b,1)〉 〈G (¬a → F b),〈¬a → F b,2〉 (¬a → F b,2)〉 R〈¬a → F b,3〉 M 〈G (¬a → F b), G (¬a → F b) (¬a → F b,3)〉 〈G (¬a → F b),〈¬a → F b,4〉 (¬a → F b,4)〉〈¬a → F b,5〉 M 〈G (¬a → F b), 3 HEIGHT (¬a → F b,5)〉
110. 110. a a b c b a ? G (¬a → F b) ⊨ 〈G (¬a → F b),0〉 〈G (¬a → F b),1〉 〈G (¬a → F b), 〈G (¬a → F b),2〉〈¬a → F b,0〉 (¬a → F b,0)〉 〈G (¬a → F b),3〉〈¬a → F b,1〉 M 〈G (¬a → F b), 〈G (¬a → F b),4〉 (¬a → F b,1)〉 〈G (¬a → F b),5〉 〈G (¬a → F b),〈¬a → F b,2〉 (¬a → F b,2)〉 R〈¬a → F b,3〉 M 〈G (¬a → F b), G (¬a → F b) (¬a → F b,3)〉 〈G (¬a → F b),〈¬a → F b,4〉 (¬a → F b,4)〉〈¬a → F b,5〉 M 〈G (¬a → F b), 3 HEIGHT (¬a → F b,5)〉
111. 111. a a b c b a ? G (¬a → F b) ⊨〈G (¬a → F b),0〉〈G (¬a → F b),1〉 W〈G (¬a → F b),2〉〈G (¬a → F b),3〉〈G (¬a → F b),4〉〈G (¬a → F b),5〉 4 HEIGHT
112. 112. a a b c b a ? G (¬a → F b) ⊨〈G (¬a → F b),0〉〈G (¬a → F b),1〉 W〈G (¬a → F b),2〉〈G (¬a → F b),3〉 True〈G (¬a → F b),4〉〈G (¬a → F b),5〉 4 HEIGHT
113. 113. The trace can be stored in� separate (and non-contiguous) chunks (a,0) (b,2) (a,1) (c,3) (a,5) (b,4) M R Mappers and reducers of aM M R R � given height can operate in parallel
114. 114. Tests on 500 randomly-generated traces From 1 to 100,000 events Each event contains 10 parameters named p₀ to p₉ with 10 possible values
115. 115. Validation of 4 LTL formulas: 1 G p₀ ≠ 0 2 G (p₀ = 0 → X p₁ = 0) 3 ∀x ∈ [0,9] : G (p₀ = x → X p₁ = x) 4 ∃m ∈ [0,9] : ∀x ∈ [0,9] : G (p m = x → X X p m ≠ x)
116. 116. Property 1 2 3 4 Tuples 55 k 120 k 600 k 5 M Time/event 19 μs 23 μs 75 μs 985 μsSequential ratio 100% 92% 92% 3% Inferred time 19 μs 21 μs 14 μs 30 μs
117. 117. Questions? M