A Case for "Piggyback" Runtime Monitoring

818 views
746 views

Published on

A runtime monitor enforcing a constraint on sequences of method calls on an object must keep track of the state of the sequence by updating an appropriate state machine. The present paper stems from the observation that an object's member fields must already contain an encoding of that state machine, and that a monitor essentially duplicates operations that the object performs internally. Rather than maintain a state machine in parallel, the paper puts forward the concept of "piggyback" runtime monitoring, where the monitor relies as much as possible on the object's own state variables to perform its task. Experiments on real-world benchmarks show that this approach greatly simplifies the monitoring process and drastically reduces the incurred runtime overhead compared to classical solutions.

2 Comments
0 Likes
Statistics
Notes
  • Be the first to like this

No Downloads
Views
Total views
818
On SlideShare
0
From Embeds
0
Number of Embeds
5
Actions
Shares
0
Downloads
2
Comments
2
Likes
0
Embeds 0
No embeds

No notes for slide

A Case for "Piggyback" Runtime Monitoring

  1. 1. THE FOLLOWING TALK HAS BEEN APPROVED FOR OPEN-MINDED AUDIENCES S SCIENTIFIC RESEARCH CONTENT CONTROVERSIAL IDEAS, CONJECTURES BACKED BY PRELIMINARY EXPERIMENTAL DATA R
  2. 2. PrologueWhat Plato taught us
  3. 3. A Case for "Piggyback" Runtime MonitoringRaphaël Tremblay-Lessard and Sylvain Hallé Laboratoire dinformatique formelleUniversité du Québec à Chicoutimi, Canada Fonds de recherche sur la nature et les technologies NSERC CRSNG
  4. 4. System
  5. 5. System
  6. 6. Instrumentation System
  7. 7. Instrumentation System
  8. 8. Instrumentation Trace System
  9. 9. Instrumentation Trace Events System
  10. 10. Instrumentation Trace Events System
  11. 11. TraceInstrumentation validation Trace Events System
  12. 12. Instrumentation System
  13. 13. Instrumentation Runtime monitoring System
  14. 14. Instrumentation Runtime monitoring System
  15. 15. Instrumentation Runtime monitoringOverhead System
  16. 16. We consider a set A of method calls on asingle instance of some class C
  17. 17. We consider a set A of method calls on asingle instance of some class CC distinguishes between “valid” and“invalid” sequences of method calls in A
  18. 18. We consider a set A of method calls on asingle instance of some class CC distinguishes between “valid” and“invalid” sequences of method calls in AC is deterministic: for every sequencem ∈ A*, m is either always valid oralways invalid
  19. 19. ( ∧ )→
  20. 20. ( ∧ )→( ∨∅)→ X
  21. 21. My car leaves
  22. 22. Mailman delivers mail
  23. 23. Neighbor picks mail
  24. 24. My car returns
  25. 25. * * *Error
  26. 26. or<T>Iterat
  27. 27. hasNext or<T> Iterat next
  28. 28. next hasNexthasNext hasNext or<T> Iterat next
  29. 29. Close Read Open for read CloseOpen for write Write
  30. 30. class MyHouse { boolean isHome = false; boolean hasMail = false; public void carLeaves() { isHome = false; } public void carReturns() { isHome = true; } public void mailman() { hasMail = true; } public void pickMail() { if (hasMail && !isHome) hasMail = false; else Do Something Bad; }}
  31. 31. There exists a minimal DFA M = 〈Q,q₀,δ〉such that ℒ(M) = all valid sequences ofmethod calls in A
  32. 32. 1 Let q = q₀2 Wait for method call m ∈ A3 Let q = δ(q, m)4 If q = then raise error5 Else goto 2
  33. 33. MyProgram: m1.carLeaves(); m2.mailman(); m1.pickMail(); . . .
  34. 34. aspect Monitor (MyHouse m) {MyProgram: int state = 0; pointcut before carLeaves() { if (state == 0) state = 1; m1.carLeaves(); } pointcut before carReturns() { if (state == 1) state = 0; m2.mailman(); } pointcut before mailman() { if (state == 1) m1.pickMail(); state = 2; } pointcut before pickMail() { if (state == 2) . . . state = 1; }
  35. 35. aspect Monitor (MyHouse m) {MyProgram: int state = 0; pointcut before carLeaves() { if (state == 0) state = 1; m1.carLeaves(); } pointcut before carReturns() { if (state == 1) state = 0; m2.mailman(); } pointcut before mailman() { if (state == 1) m1.pickMail(); state = 2; } pointcut before pickMail() { if (state == 2) . . . state = 1; }
  36. 36. Non-intrusive......but "memoryful" (must persist q)Lots of events to watch
  37. 37. OVERHEAD
  38. 38. Neighbor about to pick mail
  39. 39. Is there a car??
  40. 40. Is there mailin the box? ?
  41. 41. Key point
  42. 42. Key pointDont observemethod calls...
  43. 43. Key pointDont observe ...query the objectsmethod calls... state directly ! ? ?
  44. 44. class MyHouse { aspect Monitor (MyHouse m) { int state = 0; boolean isHome = false; boolean hasMail = false; pointcut before carLeaves() { if (state == 0) public void carLeaves() { state = 1; isHome = false; } } pointcut before carReturns() { public void carReturns() { if (state == 1) isHome = true; state = 0; } } public void mailman() { pointcut before mailman() { hasMail = true; if (state == 1) } state = 2; public void pickMail() { } if (hasMail && !isHome) pointcut before pickMail() { hasMail = false; if (state == 2) else Do Something Bad; state = 1; } }}
  45. 45. class MyHouse { aspect Monitor (MyHouse m) { boolean isHome = false; boolean hasMail = false; public void carLeaves() { isHome = false; } public void carReturns() { isHome = true; } public void mailman() { hasMail = true; } public void pickMail() { if (hasMail && !isHome) pointcut before pickMail() { hasMail = false; if (m.isHome || !m.hasMail) else Do Something Bad; // Error } }}
  46. 46. class MyHouse { aspect Monitor (MyHouse m) { boolean isHome = false; boolean hasMail = false; No state to persist public void carLeaves() { isHome = false; } Only one method to public void carReturns() { isHome = true; watch } public void mailman() { hasMail = true; Same effect ! } public void pickMail() { if (hasMail && !isHome) pointcut before pickMail() { hasMail = false; if (m.isHome || !m.hasMail) else Do Something Bad; // Error } }}
  47. 47. Key point (again)The first monitor observes method calls todeduce the objects current stateThe second monitor rather "piggybacks"on the objects own state
  48. 48. Key point (again)The first monitor observes method calls todeduce the objects current stateThe second monitor rather "piggybacks"on the objects own state PIGGYBACK RUNTIME MONITORING
  49. 49. On an Iterator i, method i.next() shouldonly be called...
  50. 50. On an Iterator i, method i.next() shouldonly be called...if it immediatelyfollows i.hasNext()
  51. 51. On an Iterator i, method i.next() shouldonly be called...if it immediately if it has more elementsfollows i.hasNext() to enumerate
  52. 52. On an Iterator i, method i.next() shouldonly be called...if it immediately if it has more elementsfollows i.hasNext() to enumerate history-based
  53. 53. On an Iterator i, method i.next() shouldonly be called...if it immediately if it has more elementsfollows i.hasNext() to enumerate history-based state-based
  54. 54. Sequences ofmethod calls
  55. 55. Sequences of method callsObjectsstate
  56. 56. Wait a minute...
  57. 57. Wait a minute... Does it work all the time?
  58. 58. Let ~ be an equivalence relation betweentwo sequences m, m ∈ A* of methodcalls
  59. 59. Let ~ be an equivalence relation betweentwo sequences m, m ∈ A* of methodcalls m ~ m ⇔ for every m ∈ A*, both mm and mm are either valid or invalid [m] = equivalence class of m
  60. 60. Let S = { [m] : m ∈ A*}.The relation ~ induces a transitionfunction ω : S × A → S ; namely ω([m],m) = [mm] C behaves like a deterministic state machine O = 〈S,[ε],ω〉
  61. 61. M aa c b a b
  62. 62. ε a [m12 O b [m12]M [m10] b a [m6] [m9]a c a a b a [m4] b [m2] a b b c a [m1] [m8] [m5] ε a b a a [m3] b [m7] [m [m0] a b c [m11] ε
  63. 63. ε a [m12 O b [m12]M [m10] b a [m6] [m9]a c a a b a [m4] b [m2] a b b c a [m1] [m8] [m5] ε a b a a [m3] b [m7] [m [m0] a b c [m11] ε
  64. 64. ε a [m12 O b [m12]M [m10] bAll states of O a [m9] [m6]are accounted fora a a c b a [m4] b [m2] a b bNo state of O c ahas two colors [m1] [m8] [m5] ε a b a a [m3] b [m7] [m [m0] a b c [m11] ε
  65. 65. ε a [m12 O b [m12]M [m10] bAll states of O a [m9] [m6]are accounted fora a a c(δ bis total) a [m4] b [m2] a b bNo state of O c ahas two colors [m1] [m8] [m5] ε a(otherwise O and M b adisagree on some m) a [m3] b [m7] [m [m0] a b c [m11] ε
  66. 66. ε a [m12 O b [m12]M [m10] bAll states of O a There exists a [m ]are accounted fora mapping ] a [m a 6 9 c(δ bis total) H : S→Q a [m ] b [m ] a such that H is a b 4 2 bNo state of O graph c ahas two colors [m ] homomorphism ] 1 [m ] [m a 8 5(otherwise O and M 〈S,ω〉 →b〈Q,δ〉 a εdisagree on some m) (just take the color!) a [m ] 3 b [m7] [m [m0] a b c [m11] ε
  67. 67. 1 Wait for method call m ∈ A2 Fetch s3 Compute q = δ(H(s), m)4 If q = then raise error5 Else goto 1
  68. 68. 1 Wait for method call m ∈ A2 Fetch s3 Compute q = δ(H(s), m)4 If q = then raise error5 Else goto 1 Memory-less (nothing to persist between calls) Only monitor calls that may lead to
  69. 69. Collection of 13 different monitoringproperties in papers from 2001-2011(java.util package: the “monitoring canon”)
  70. 70. Collection of 13 different monitoringproperties in papers from 2001-2011(java.util package: the “monitoring canon”) Given an Iterator i on a Collection c, i.next() cannot follow any of c.add(), c.delete(), c.set() Given an Iterator i, i.remove() should not be called twice without a call to i.next() in between Don’t use an InputStreamReader after its InputStream was closed ...
  71. 71. Source code analysis of the OpenJDK 6implementation of Java. Of the 13 propertiesin the canon:
  72. 72. Source code analysis of the OpenJDK 6implementation of Java. Of the 13 propertiesin the canon: 10 can be piggyback-monitored
  73. 73. Source code analysis of the OpenJDK 6implementation of Java. Of the 13 propertiesin the canon: 10 can be piggyback-monitored Remaining 3 either...
  74. 74. Source code analysis of the OpenJDK 6implementation of Java. Of the 13 propertiesin the canon: 10 can be piggyback-monitored Remaining 3 either... dont distinguish between valid/invalid calls
  75. 75. Source code analysis of the OpenJDK 6implementation of Java. Of the 13 propertiesin the canon: 10 can be piggyback-monitored Remaining 3 either... dont distinguish between valid/invalid calls violate deterministic condition
  76. 76. Wait a minute...
  77. 77. Wait a minute... How hard is it to obtain H and s?
  78. 78. The graph homomorphismproblem
  79. 79. The graph homomorphismproblemFinding an homomorphism H betweentwo directed graphs G and G is NP-complete.
  80. 80. The graph homomorphismproblemFinding an homomorphism H betweentwo directed graphs G and G is NP-complete. (Known result)
  81. 81. The graph homomorphismproblemFinding an homomorphism H betweentwo directed graphs G and G is NP-complete. (Known result)⇒ Checking if a known H “works” is polynomial
  82. 82. Wait a minute...
  83. 83. cla ss int C B b x;... ; Wait a minute...
  84. 84. cla ss flo B a A a t w;cla int ; ss D d j; int C ; B b x;... ; Wait a minute...
  85. 85. cla ss cla flo A ss a int t z; flo B k; a A a t w;cla int ; ss D d j; int C ; B b x;... ; Wait a minute...
  86. 86. cla ss cla flo A ss a int t z; flo B k; a A a t w;cla int ; ss D d j; int C ; B b x; cla ss... ; ... D Wait a minute...
  87. 87. cla ss cla flo A ss a int t z; flo B k; a A a t w;cla int ; ss D d j; int C ; B b x; cla ss... ; ... D Wait a minute...
  88. 88. cla ss cla flo A ss a int t z; flo B k; a A a t w;cla int ; ss D d j; int C ; B b x; cla ss... ; ... D Wait a minute... How many fields are involved?
  89. 89. Propert y REMOVE
  90. 90. Propert y REMOVE Traditional version Given an Iterator i, i.remove() should not be called twice without a call to i.next() in between
  91. 91. Propert y REMOVE Traditional version Given an Iterator i, i.remove() should not be called twice without a call to i.next() in between Piggyback version Given an Iterator i, in every call to i.remove(), it must hold that i.lastRet must not be equal to −1
  92. 92. Propert y REMOVE Traditional version Given an Iterator i, i.remove() should not be called twice without a call to i.next() in between Piggyback version 1 Given an Iterator i, in every call to member field i.remove(), it must hold that i.lastRet involved must not be equal to −1
  93. 93. Propert y SAFEENUM
  94. 94. Propert y SAFEENUM Traditional version Given an Iterator i on a Collection c, i.next() cannot follow any of c.add(), c.delete(), c.set(), etc.
  95. 95. Propert y SAFEENUM Traditional version Given an Iterator i on a Collection c, i.next() cannot follow any of c.add(), c.delete(), c.set(), etc. Piggyback version Given an Itr i, in every call to i.next(), it must hold that i.expectedModCount is equal to i$0.modCount
  96. 96. Propert y SAFEENUM Traditional version Given an Iterator i on a Collection c, i.next() cannot follow any of c.add(), c.delete(), c.set(), etc. Piggyback version 2 Given an Itr i, in every call to i.next(), member fields it must hold that i.expectedModCount is involved equal to i$0.modCount
  97. 97. With the OpenJDK 6, the 10 piggybackproperties involved at most _______primitive member fields inside a class
  98. 98. With the OpenJDK 6, the 10 piggyback 2properties involved at most _______primitive member fields inside a class
  99. 99. With the OpenJDK 6, the 10 piggyback 2properties involved at most _______primitive member fields inside a classat a level of nesting of at most _______
  100. 100. With the OpenJDK 6, the 10 piggyback 2properties involved at most _______primitive member fields inside a class 1at a level of nesting of at most _______
  101. 101. With the OpenJDK 6, the 10 piggyback 2properties involved at most _______primitive member fields inside a class 1at a level of nesting of at most _______ H is simple and shallow
  102. 102. With the OpenJDK 6, the 10 piggyback 2properties involved at most _______primitive member fields inside a class 1at a level of nesting of at most _______ H is simple and shallow ...but many fields are private and have no accessor methods ☹
  103. 103. Propert y SAFEF ILE READER Traditional version Don’t use an InputStreamReader after its InputStream was closed Piggyback version Given an InputStreamReader r, in every call to r.read(), it must hold that r.sd.isOpen is true
  104. 104. Propert y SAFEF ILE READER Traditional version Don’t use an InputStreamReader after its InputStream was closed Piggyback version Given an InputStreamReader r, in every call to r.read(), it must hold that r.sd.isOpen is true private
  105. 105. class InputStreamReader { private InputStream sd; public boolean isStreamOpen() { return sd.isOpen(); } class InputStream { ... private boolean isOpen;} public boolean isOpen() { return isOpen; } ... }
  106. 106. class InputStreamReader { private InputStream sd; public boolean isStreamOpen() { return sd.isOpen(); } MISSING class InputStream { ... private boolean isOpen;} public boolean isOpen() { return isOpen; } ... }
  107. 107. class InputStreamReader { private InputStream sd; public boolean isStreamOpen() { return sd.isOpen(); } MISSING class InputStream { ... private boolean isOpen;} public boolean isOpen() { return isOpen; } MISSING ... }
  108. 108. cla ss cla flo A ss a int t z; flo B k; a A a t w;cla int ; ss D d j; int C ; B b x; cla ss... ; ... D Wait a minute...
  109. 109. cla ss cla pri vat flo A ss e i at z flo B nt ; pri a k; v A a t w;pri cla pri ate iint ; vat ss vat e nt D d j; ei C D ; int nt xx; Bb ; cla ss ... ; ... D Wait a minute... What about information hiding?
  110. 110. “...one begins [to decompose a system] with alist of difficult design decisions or designdecisions that are likely to change. Eachmodule is then designed to hide such adecision from the others.” ― David Lodge Parnas, 1972
  111. 111. class InputStreamReader { private InputStream sd; public boolean isStreamOpen() { return sd.isOpen(); } class InputStream { ... private boolean isOpen;} public boolean isOpen() { return isOpen; } ... }
  112. 112. class InputStreamReader { private InputStream sd; public boolean isStreamOpen() { return sd.isOpen(); } class InputStream { ... private boolean isOpen;} public boolean isOpen() { return isOpen; Difficult design decision? } Likely to change? ... }
  113. 113. The member fields of C involved inthe homomorphism H should havethe same read access visibilit y as C
  114. 114. design for monitoring The member fields of C involved in the homomorphism H should have the same read access visibilit y as C
  115. 115. design for monitoring (or just for correct use) The member fields of C involved in the homomorphism H should have the same read access visibilit y as C
  116. 116. Wait a minute...
  117. 117. OVERHEADWait a minute... Is overhead actually reduced ?
  118. 118. Runtime monitoring of the 10 piggybackproperties on the DaCapo benchmark Set of open source, real world applications with non-trivial memory load Same benchmark used by past monitoring papers
  119. 119. Benchmark error margin ≈ 3%
  120. 120. Benchmark error margin ≈ 3%Piggyback memory overhead ≈ 0%
  121. 121. Benchmark error margin ≈ 3%Piggyback memory overhead ≈ 0%Piggyback runtime overhead ≈ 3%
  122. 122. Benchmark error margin ≈ 3%Piggyback memory overhead ≈ 0%Piggyback runtime overhead ≈ 3%Highlights: Runtime overhead (%) Benchmark Piggyback pmd ≈ 0% avrora ≈ 1%
  123. 123. Benchmark error margin ≈ 3%Piggyback memory overhead ≈ 0%Piggyback runtime overhead ≈ 3%Highlights: Runtime overhead (%) Benchmark Classical Piggyback pmd ≈ 123% ≈ 0% avrora ≈ 118% ≈ 1%
  124. 124. EpilogueConclusion and future work
  125. 125. Piggyback runtime monitoring uses anobjects own, existing member fields as anencoding of the monitors stateUnder some conditions, this encoding isguaranteed to existStateful properties become statelessproperties ; no data to persist between calls
  126. 126. Emprical evidence on real-worldbenchmarks on the OpenJDK 6 revealthat:
  127. 127. Emprical evidence on real-worldbenchmarks on the OpenJDK 6 revealthat: Most properties are piggyback- compatible
  128. 128. Emprical evidence on real-worldbenchmarks on the OpenJDK 6 revealthat: Most properties are piggyback- compatible H involves few variables
  129. 129. Emprical evidence on real-worldbenchmarks on the OpenJDK 6 revealthat: Most properties are piggyback- compatible H involves few variables Runtime overhead is reduced over classical monitors
  130. 130. Future work and open questions: Use annotations instead of aspects Find H automatically Verify a given H statically Extend to methods with arguments Relax hypotheses
  131. 131. Piggyback and classical monitors arecomplementary
  132. 132. Piggyback and classical monitors arecomplementaryThe choice of a classical monitor should bejustified, not taken for granted
  133. 133. Piggyback and classical monitors arecomplementaryThe choice of a classical monitor should bejustified, not taken for grantedIs overhead really what matters?
  134. 134. The EndThank you !

×