Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

MapReduce for Parallel Trace Validation of LTL Properties

1,227 views

Published on

We present an algorithm for the automated verification of Linear Temporal Logic formulae on event traces using an increasingly popular cloud computing framework called MapReduce. The algorithm can process multiple, arbitrary fragments of the trace in parallel, and compute its final result through a cycle of runs of MapReduce instances. Compared to classical, single-instance solutions, a proof-of-concept implementation shows through experimental evaluation how the algorithm reduces by as much as 90% the number of operations that must be performed linearly, resulting in a commensurate speed gain.

Published in: Technology, Business
  • Be the first to comment

  • Be the first to like this

MapReduce for Parallel Trace Validation of LTL Properties

  1. 1. MapReduce for ParallelTrace Validation of LTL PropertiesBenjamin Barre, Mathieu Klein, Maxime Soucy-Boivin, Pierre-Antoine Ollivier and Sylvain Hallé Université du Québec à Chicoutimi CANADA Fonds de recherche Nature et NSERC technologies CRSNG
  2. 2. System
  3. 3. System
  4. 4. Instrumentation System
  5. 5. Instrumentation System
  6. 6. Instrumentation Trace System
  7. 7. Instrumentation Trace Events System
  8. 8. Instrumentation Trace Events System
  9. 9. Trace validationInstrumentation Trace Events System
  10. 10. or<T>Iterat
  11. 11. hasNext or<T> Iterat next
  12. 12. A call to next must be preceded by a call to hasNexthasNext or<T> Iterat next
  13. 13. AB
  14. 14. No CartCreate request can occurbefore a LoginResponse message A B
  15. 15. Login
  16. 16. Three successive login attemptsshould trigger an alarm Login
  17. 17. Receive order
  18. 18. Ready?Receive order
  19. 19. Ready? YesReceive order
  20. 20. Ready? YesReceive order No Ship File order
  21. 21. A received order must eventually be shipped Ready? YesReceive order No Ship File order
  22. 22. Let A be a set of event symbols.A trace m is a mapping from ℕ tothe set of events :ℕ 0 1 2 3 4 ...A a a b c b
  23. 23. X next ¬ ∧ → G globally A + ∧¬→ + F eventually U untilGround Boolean Temporal terms connectives operators = Linear Temporal Logic
  24. 24. Let Φ be the set of all possible LTL formulas.The function ℒ : Φ → 2ℕ labels each state witha set of LTL formulasℕ 0 1 2 3 4 ...A a a b c b
  25. 25. Let Φ be the set of all possible LTL formulas.The function ℒ : Φ → 2ℕ labels each state witha set of LTL formulasℒ b) ∨c b) b b b (a→ (a→ a∧c b a∧ a∧ b∨ G Gℕ 0 1 2 3 4 ...A a a b c b Example: ℒ(a∧b) = {0,1,4,...}
  26. 26. i ∈ ℒ(a) ⇔ m(i) = ai ∈ ℒ(¬φ) ⇔ i ∉ ℒ(φ)i ∈ ℒ(φ∧ψ) ⇔ i ∈ ℒ(φ) and i ∈ ℒ(ψ)i ∈ ℒ(φ∨ψ) ⇔ i ∈ ℒ(φ) or i ∈ ℒ(ψ)i ∈ ℒ(X φ) ⇔ i+1 ∈ ℒ(φ)i ∈ ℒ(G φ) ⇔ j ∈ ℒ(φ) for all j ≥ ii ∈ ℒ(F φ) ⇔ j ∈ ℒ(φ) for some j ≥ ii ∈ ℒ(φ U ψ) ⇔ j ∈ ℒ(ψ) for some j ≥ i and k ∈ ℒ(φ) for all j ≥ k ≥ i
  27. 27. Theorem i ∈ ℒ(φ) exactly when the trace m(i), m(i+1), ... satisfies φ σ φ ψ 0 1 2 3 4 ...
  28. 28. Theorem i ∈ ℒ(φ) exactly when the trace m(i), m(i+1), ... satisfies φ σ φ ψ 0 1 2 3 4 ... Therefore... 0 ∈ ℒ(φ) ⇔ m ⊧ φ
  29. 29. A call to next must be followed by a callto hasNextNo CartCreate request can occurbefore a LoginResponse messageA received order must eventuallybe shippedThree successive login attempts shouldtrigger an alarm
  30. 30. A call to next must be followed by a callto hasNext G (next → X hasNext)No CartCreate request can occurbefore a LoginResponse messageA received order must eventuallybe shippedThree successive login attempts shouldtrigger an alarm
  31. 31. A call to next must be followed by a callto hasNext G (next → X hasNext)No CartCreate request can occurbefore a LoginResponse message ¬ CartCreate U hasNextA received order must eventuallybe shippedThree successive login attempts shouldtrigger an alarm
  32. 32. A call to next must be followed by a callto hasNext G (next → X hasNext)No CartCreate request can occurbefore a LoginResponse message ¬ CartCreate U hasNextA received order must eventuallybe shipped G (receive → F ship)Three successive login attempts shouldtrigger an alarm
  33. 33. A call to next must be followed by a callto hasNext G (next → X hasNext)No CartCreate request can occurbefore a LoginResponse message ¬ CartCreate U hasNextA received order must eventuallybe shipped G (receive → F ship)Three successive login attempts shouldtrigger an alarm G ¬(fail ∧ (X (fail ∧ X fail)))
  34. 34. Iterat or<T> Java MOP
  35. 35. � The trace must be read linearly 2 3 4 5 1 � The algorithm works on a x1 single process / core / site
  36. 36. 10,000,000 1,000,000 100,000 10,000 1,000 100 10 1 1970 1980 1990 2000 2010
  37. 37. 10,000,000 Transistors (x1000) 1,000,000 100,000 10,000 1,000 100 10 1 1970 1980 1990 2000 2010
  38. 38. 10,000,000 Transistors (x1000) 1,000,000 100,000 10,000 CPU Speed 1,000 (MHz) 100 10 1 1970 1980 1990 2000 2010
  39. 39. PageRank f ∞
  40. 40. ValueKey a 1{ Tuple (baaah)
  41. 41. Data source
  42. 42. Data sourceIIInput reader
  43. 43. Data sourceII a 2 z 7 . . .Input reader
  44. 44. . . . a 2 a 2
  45. 45. Mapper. . . a 2 a 2 M
  46. 46. Mapper. . . a 2 a 2 M w 6 a 2 . . .
  47. 47. Mapper. . . a 2 a 2 M w 6 a 2 . . . a b3 a2 3 3 g b
  48. 48. a 3 a3 2a e 38 a a3 b 9 bb a 3 ba a
  49. 49. e 38 a b . . . . . . b db Shuffling aa ba ab a a 2 a b 9 3
  50. 50. b 9 ba a 2 2a 3 a
  51. 51. b 9 ba a 2 2a 3 a Rb Ra Reducer
  52. 52. b 9 ba a 2 2a 3 a Rb e 7 i 0 Ra z 8 x 2 . . . Reducer
  53. 53. a b a a b a
  54. 54. a b a a b aa aa bb a
  55. 55. a b a a b aa a Ia bb a
  56. 56. a b a a b a 〈a,1〉a a I 〈a,1〉a bb a
  57. 57. a b a a b a 〈a,1〉a a I 〈a,1〉 〈a,1〉a b I 〈b,1〉 〈b,1〉b a I 〈a,1〉
  58. 58. a b a a b a 〈a,1〉a a I 〈a,1〉 〈a,1〉a b I 〈b,1〉 〈b,1〉b a I 〈a,1〉
  59. 59. a b a a b a 〈a,1〉a a I 〈a,1〉 〈a,1〉a b I 〈b,1〉 〈b,1〉b a I 〈a,1〉
  60. 60. a b a a b a 〈a,1〉a a I 〈a,1〉 Ra 〈a,1〉a b I 〈b,1〉 〈b,1〉b a I 〈a,1〉
  61. 61. a b a a b a 〈a,1〉a a I 〈a,1〉 Ra 〈a,4〉 〈a,1〉a b I 〈b,1〉 〈b,1〉b a I 〈a,1〉
  62. 62. a b a a b a 〈a,1〉a a I 〈a,1〉 Ra 〈a,4〉 〈a,1〉a b I 〈b,1〉 〈b,1〉 Rb 〈b,2〉b a I 〈a,1〉
  63. 63. a b a a b a 〈a,1〉a a I 〈a,1〉 Ra 〈a,4〉 〈a,1〉a b I 〈b,1〉 〈b,1〉 Rb 〈b,2〉b a I 〈a,1〉
  64. 64. Superformula SuperformulaFormula G ∧ Subformula Subformula Subformula
  65. 65. 4 GHeight 3 → 2 ∧ 3 1 2 ¬ F 1 0 0 a c b
  66. 66. 4 GHeight 3 → 2 ∧ 3 1 2 ¬ F 1 0 0 a c b¬c has height 1G ((a ∧¬c) → F b) has height 4
  67. 67. i ∈ ℒ(a) ⇔ m(i) = ai ∈ ℒ(¬φ) ⇔ i ∉ ℒ(φ)i ∈ ℒ(φ∧ψ) ⇔ i ∈ ℒ(φ) and i ∈ ℒ(ψ)i ∈ ℒ(φ∨ψ) ⇔ i ∈ ℒ(φ) or i ∈ ℒ(ψ)i ∈ ℒ(X φ) ⇔ i+1 ∈ ℒ(φ)i ∈ ℒ(G φ) ⇔ j ∈ ℒ(φ) for all j ≥ ii ∈ ℒ(F φ) ⇔ j ∈ ℒ(φ) for some j ≥ ii ∈ ℒ(φ U ψ) ⇔ j ∈ ℒ(ψ) for some j ≥ i and k ∈ ℒ(φ) for all j ≥ k ≥ i
  68. 68. i ∈ ℒ(a) ⇔ m(i) = a i ∈ ℒ(¬φ) ⇔ i ∉ ℒ(φ) i ∈ ℒ(φ∧ψ) ⇔ i ∈ ℒ(φ) and i ∈ ℒ(ψ) i ∈ ℒ(φ∨ψ) ⇔ i ∈ ℒ(φ) or i ∈ ℒ(ψ) i ∈ ℒ(X φ) ⇔ i+1 ∈ ℒ(φ) i ∈ ℒ(G φ) ⇔ j ∈ ℒ(φ) for all j ≥ i i ∈ ℒ(F φ) ⇔ j ∈ ℒ(φ) for some j ≥ i i ∈ ℒ(φ U ψ) ⇔ j ∈ ℒ(ψ) for some j ≥ i and k ∈ ℒ(φ) for all j ≥ k ≥ iThe labelling of a formula depends onlyon labellings of formulas of strictly lower height
  69. 69. i ∈ ℒ(a) ⇔ m(i) = a i ∈ ℒ(¬φ) ⇔ i ∉ ℒ(φ) i ∈ ℒ(φ∧ψ) ⇔ i ∈ ℒ(φ) and i ∈ ℒ(ψ) i ∈ ℒ(φ∨ψ) ⇔ i ∈ ℒ(φ) or i ∈ ℒ(ψ) i ∈ ℒ(X φ) ⇔ i+1 ∈ ℒ(φ) i ∈ ℒ(G φ) ⇔ j ∈ ℒ(φ) for all j ≥ i i ∈ ℒ(F φ) ⇔ j ∈ ℒ(φ) for some j ≥ i i ∈ ℒ(φ U ψ) ⇔ j ∈ ℒ(ψ) for some j ≥ i and k ∈ ℒ(φ) for all j ≥ k ≥ iThe labelling of a formula depends onlyon labellings of formulas of strictly lower height⇒ All labellings of formulas of same height are independent
  70. 70. i ∈ ℒ(a) ⇔ m(i) = a i ∈ ℒ(¬φ) ⇔ i ∉ ℒ(φ) i ∈ ℒ(φ∧ψ) ⇔ i ∈ ℒ(φ) and i ∈ ℒ(ψ) i ∈ ℒ(φ∨ψ) ⇔ i ∈ ℒ(φ) or i ∈ ℒ(ψ) i ∈ ℒ(X φ) ⇔ i+1 ∈ ℒ(φ) i ∈ ℒ(G φ) ⇔ j ∈ ℒ(φ) for all j ≥ i i ∈ ℒ(F φ) ⇔ j ∈ ℒ(φ) for some j ≥ i i ∈ ℒ(φ U ψ) ⇔ j ∈ ℒ(ψ) for some j ≥ i and k ∈ ℒ(φ) for all j ≥ k ≥ iThe labelling of a formula depends onlyon labellings of formulas of strictly lower height⇒ All labellings of formulas of same height are independent ⇒ They can be computed in parallel
  71. 71. M
  72. 72. M Input: tuples 〈φ,(n,i)〉
  73. 73. M Input: tuples 〈φ,(n,i)〉 “ n ∈ ℒ(φ), and the last cycle has evaluated labellings for formulas of height i ”
  74. 74. “Lift” ℒ(φ) to superformulas of φM Input: tuples 〈φ,(n,i)〉 “ n ∈ ℒ(φ), and the last cycle has evaluated labellings for formulas of height i ”
  75. 75. Output: tuples 〈ψ,(φ,n,i)〉 “Lift” ℒ(φ) to superformulas of φM Input: tuples 〈φ,(n,i)〉 “ n ∈ ℒ(φ), and the last cycle has evaluated labellings for formulas of height i ”
  76. 76. Output: tuples 〈ψ,(φ,n,i)〉 “ n ∈ ℒ(φ), the last cycle has evaluated labellings for formulas of height i, and φ is a subformula of ψ ” “Lift” ℒ(φ) to superformulas of φM Input: tuples 〈φ,(n,i)〉 “ n ∈ ℒ(φ), and the last cycle has evaluated labellings for formulas of height i ”
  77. 77.
  78. 78. Input:〈ψ,(φ,n,i)〉 Rψ
  79. 79. Input:〈ψ,(φ,n,i)〉 Rψ“ n ∈ ℒ(φ), the last cyclehas evaluated labellings forformulas of height i, andφ is a subformula of ψ ”
  80. 80. Input: Compute ℒ(ψ)〈ψ,(φ,n,i)〉 Rψ“ n ∈ ℒ(φ), the last cyclehas evaluated labellings forformulas of height i, andφ is a subformula of ψ ”
  81. 81. Input: Compute ℒ(ψ) Output:〈ψ,(φ,n,i)〉 〈ψ,(n,i+1)〉 Rψ“ n ∈ ℒ(φ), the last cyclehas evaluated labellings forformulas of height i, andφ is a subformula of ψ ”
  82. 82. Input: Compute ℒ(ψ) Output:〈ψ,(φ,n,i)〉 〈ψ,(n,i+1)〉 Rψ“ n ∈ ℒ(φ), the last cycle “ n ∈ ℒ(ψ), and the lasthas evaluated labellings for cycle has evaluatedformulas of height i, and labellings for formulas ofφ is a subformula of ψ ” height i+1
  83. 83. I
  84. 84. Input: events (a,n) I
  85. 85. Input: events (a,n) Output: tuples 〈ψ,(a,n,0)〉 I . . . “ n ∈ ℒ(a), the last cycle has evaluated labellings for formulas of height 0, and a is a subformula of ψ ”
  86. 86. W
  87. 87. Input: 〈ψ,(n,i)〉 W
  88. 88. Output:Input: 〈ψ,(n,i)〉 True if 〈ψ,(0,i)〉 is read W False otherwise
  89. 89. 3 . . . R W R 2 R R R 1 I . . . RI R R
  90. 90. 3 . . . R W R 2 R R R 1 I . . . R I R RInputReaders generate the first tuples fromthe trace chunks
  91. 91. 3 . . . R W R 2 R R R 1 I . . . R I R RThe tuples are shuffled to reducers that compute thelabelling ℒ for formulas of height 1
  92. 92. 3 . . . R W R 2 R R R 1 I . . . R I R RMappers copy the labellings into tuples marked bysuperformulas of height 2
  93. 93. 3 . . . R W R 2 R R R 1 I . . . R I R REach reducer computes the labelling of a formula ofheight 2 from the labelling of its subformulas
  94. 94. 3 . . . R W R 2 R R R 1 I . . . R I R RMappers copy the labellings into tuples marked bysuperformulas of height 3
  95. 95. 3 . . . R W R 2 R R R 1 I . . . R I R REach reducer computes the labelling of a formula ofheight 3 from the labelling of its subformulas
  96. 96. 3 . . . R W R 2 R R R 1 I . . . R I R RAn output writer collects the resulting tuples, andoutputs “true” if it encounters a tuple for state 0
  97. 97. a a b c b a ? G (¬a → F b) ⊨
  98. 98. a a b c b a ? G (¬a → F b) ⊨(a,0) (b,2)(a,1) (c,3)(a,5) (b,4) 0 HEIGHT
  99. 99. a a b c b a ? G (¬a → F b) ⊨(a,0) (b,2) I(a,1) (c,3) I(a,5) (b,4) I 0 HEIGHT
  100. 100. a a b c b a ? G (¬a → F b) ⊨ 〈¬a,(a,0)〉(a,0) (b,2) I 〈F b,(b,2)〉(a,1) (c,3) I 〈¬a,(a,1)〉 〈F b,(b,4)〉(a,5) (b,4) I 0 1 〈¬a,(a,5)〉 HEIGHT
  101. 101. a a b c b a ? G (¬a → F b) ⊨ 〈¬a,(a,0)〉(a,0) (b,2) I 〈F b,(b,2)〉 R ¬a(a,1) (c,3) I 〈¬a,(a,1)〉 R 〈F b,(b,4)〉 Fb(a,5) (b,4) I 0 1 〈¬a,(a,5)〉 HEIGHT
  102. 102. a a b c b a ? G (¬a → F b) ⊨ 〈¬a,(a,0)〉 〈¬a,2〉(a,0) (b,2) I 〈¬a,3〉 〈F b,(b,2)〉 R ¬a 〈¬a,4〉(a,1) (c,3) I 〈¬a,(a,1)〉 〈F b,0〉 〈F b,1〉 R 〈F b,2〉 〈F b,(b,4)〉 Fb 〈F b,3〉(a,5) (b,4) I 0 1 〈¬a,(a,5)〉 b,4〉 〈FHEIGHT
  103. 103. a a b c b a ? G (¬a → F b) ⊨〈¬a,2〉〈¬a,3〉 M〈¬a,4〉〈F b,0〉〈F b,1〉 M〈F b,2〉〈F b,3〉〈F b,4〉 M 2 HEIGHT
  104. 104. a a b c b a ? G (¬a → F b) ⊨〈¬a,2〉 〈¬a → F b,(¬a,2)〉〈¬a,3〉 M 〈¬a → F b,(¬a,3)〉〈¬a,4〉 〈¬a → F b,(¬a,4)〉〈F b,0〉 〈¬a → F b,(F b,0)〉〈F b,1〉 〈¬a → F b,(F b,1)〉 M 〈¬a → F b,(F b,2)〉〈F b,2〉 〈¬a → F b,(F b,3)〉〈F b,3〉〈F b,4〉 M 〈¬a → F b,(F b,4)〉 2 HEIGHT
  105. 105. a a b c b a ? G (¬a → F b) ⊨〈¬a,2〉 〈¬a → F b,(¬a,2)〉〈¬a,3〉 M 〈¬a → F b,(¬a,3)〉〈¬a,4〉 〈¬a → F b,(¬a,4)〉〈F b,0〉 〈¬a → F b,(F b,0)〉〈F b,1〉 〈¬a → F b,(F b,1)〉 R M 〈¬a → F b,(F b,2)〉 ¬a → Fb〈F b,2〉 〈¬a → F b,(F b,3)〉〈F b,3〉〈F b,4〉 M 〈¬a → F b,(F b,4)〉 2 HEIGHT
  106. 106. a a b c b a ? G (¬a → F b) ⊨ 〈¬a → F b,0〉 〈¬a → F b,1〉 〈¬a → F b,2〉〈¬a,2〉 〈¬a → F b,(¬a,2)〉 〈¬a → F b,3〉〈¬a,3〉 M 〈¬a → F b,(¬a,3)〉 〈¬a → F b,4〉〈¬a,4〉 〈¬a → F b,(¬a,4)〉 〈¬a → F b,5〉〈F b,0〉 〈¬a → F b,(F b,0)〉〈F b,1〉 〈¬a → F b,(F b,1)〉 R M 〈¬a → F b,(F b,2)〉 ¬a → Fb〈F b,2〉 〈¬a → F b,(F b,3)〉〈F b,3〉〈F b,4〉 M 〈¬a → F b,(F b,4)〉 2 HEIGHT
  107. 107. a a b c b a ? G (¬a → F b) ⊨〈¬a → F b,0〉〈¬a → F b,1〉 M〈¬a → F b,2〉〈¬a → F b,3〉 M〈¬a → F b,4〉〈¬a → F b,5〉 M 3 HEIGHT
  108. 108. a a b c b a ? G (¬a → F b) ⊨ 〈G (¬a → F b),〈¬a → F b,0〉 (¬a → F b,0)〉〈¬a → F b,1〉 M 〈G (¬a → F b), (¬a → F b,1)〉 〈G (¬a → F b),〈¬a → F b,2〉 (¬a → F b,2)〉〈¬a → F b,3〉 M 〈G (¬a → F b), (¬a → F b,3)〉 〈G (¬a → F b),〈¬a → F b,4〉 (¬a → F b,4)〉〈¬a → F b,5〉 M 〈G (¬a → F b), 3 HEIGHT (¬a → F b,5)〉
  109. 109. a a b c b a ? G (¬a → F b) ⊨ 〈G (¬a → F b),〈¬a → F b,0〉 (¬a → F b,0)〉〈¬a → F b,1〉 M 〈G (¬a → F b), (¬a → F b,1)〉 〈G (¬a → F b),〈¬a → F b,2〉 (¬a → F b,2)〉 R〈¬a → F b,3〉 M 〈G (¬a → F b), G (¬a → F b) (¬a → F b,3)〉 〈G (¬a → F b),〈¬a → F b,4〉 (¬a → F b,4)〉〈¬a → F b,5〉 M 〈G (¬a → F b), 3 HEIGHT (¬a → F b,5)〉
  110. 110. a a b c b a ? G (¬a → F b) ⊨ 〈G (¬a → F b),0〉 〈G (¬a → F b),1〉 〈G (¬a → F b), 〈G (¬a → F b),2〉〈¬a → F b,0〉 (¬a → F b,0)〉 〈G (¬a → F b),3〉〈¬a → F b,1〉 M 〈G (¬a → F b), 〈G (¬a → F b),4〉 (¬a → F b,1)〉 〈G (¬a → F b),5〉 〈G (¬a → F b),〈¬a → F b,2〉 (¬a → F b,2)〉 R〈¬a → F b,3〉 M 〈G (¬a → F b), G (¬a → F b) (¬a → F b,3)〉 〈G (¬a → F b),〈¬a → F b,4〉 (¬a → F b,4)〉〈¬a → F b,5〉 M 〈G (¬a → F b), 3 HEIGHT (¬a → F b,5)〉
  111. 111. a a b c b a ? G (¬a → F b) ⊨〈G (¬a → F b),0〉〈G (¬a → F b),1〉 W〈G (¬a → F b),2〉〈G (¬a → F b),3〉〈G (¬a → F b),4〉〈G (¬a → F b),5〉 4 HEIGHT
  112. 112. a a b c b a ? G (¬a → F b) ⊨〈G (¬a → F b),0〉〈G (¬a → F b),1〉 W〈G (¬a → F b),2〉〈G (¬a → F b),3〉 True〈G (¬a → F b),4〉〈G (¬a → F b),5〉 4 HEIGHT
  113. 113. The trace can be stored in� separate (and non-contiguous) chunks (a,0) (b,2) (a,1) (c,3) (a,5) (b,4) M R Mappers and reducers of aM M R R � given height can operate in parallel
  114. 114. Tests on 500 randomly-generated traces From 1 to 100,000 events Each event contains 10 parameters named p₀ to p₉ with 10 possible values
  115. 115. Validation of 4 LTL formulas: 1 G p₀ ≠ 0 2 G (p₀ = 0 → X p₁ = 0) 3 ∀x ∈ [0,9] : G (p₀ = x → X p₁ = x) 4 ∃m ∈ [0,9] : ∀x ∈ [0,9] : G (p m = x → X X p m ≠ x)
  116. 116. Property 1 2 3 4 Tuples 55 k 120 k 600 k 5 M Time/event 19 μs 23 μs 75 μs 985 μsSequential ratio 100% 92% 92% 3% Inferred time 19 μs 21 μs 14 μs 30 μs
  117. 117. Questions? M

×