A Runtime Monitoring Framework for Event Streams with Non-Primitive Arguments
Upcoming SlideShare
Loading in...5
×
 

A Runtime Monitoring Framework for Event Streams with Non-Primitive Arguments

on

  • 478 views

A runtime monitor is a tool that takes as input a model of some system, and observes in real time that the sequence of events produced by a run of that system follows the specification. While existing ...

A runtime monitor is a tool that takes as input a model of some system, and observes in real time that the sequence of events produced by a run of that system follows the specification. While existing monitoring solutions generally use finite-state machines and temporal logic as their model language, the specification is ultimately tangled with hand-written, implementation-specific details which severely limit their range of application. We present a runtime monitoring platform that clearly separates the extraction of events in the running program from the specification and monitoring process. This separation allows one to cleanly monitor first-order properties involving arbitrarily complex native program objects, while still incurring reasonable overhead.

Statistics

Views

Total Views
478
Slideshare-icon Views on SlideShare
478
Embed Views
0

Actions

Likes
0
Downloads
6
Comments
0

0 Embeds 0

No embeds

Accessibility

Categories

Upload Details

Uploaded via as Adobe PDF

Usage Rights

© All Rights Reserved

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Processing…
Post Comment
Edit your comment

    A Runtime Monitoring Framework for Event Streams with Non-Primitive Arguments A Runtime Monitoring Framework for Event Streams with Non-Primitive Arguments Presentation Transcript

    • A Runtime Monitoring Framework for Event Streams with Non-Primitive Arguments Jérôme Calvar, Raphaël Tremblay- Lessard, Sylvain Hallé Université du Québec à Chicoutimi CANADA Presented by Roger Villemaire Université du Québec à Montréal Fonds de recherche sur la nature et les technologies NSERC CRSNGCalvar et al.
    • Presentation overview 1. What is runtime monitoring? 2. Current issues in runtime monitoring 3. Our runtime monitoring framework 4. Experimental resultsCalvar et al.
    • What is runtime monitoring? Runtime monitoring is the process of observing an actual run of a system and dynamically checking and enforcing constraints on its execution System Monitor Specification } Event Verifies that Gives conditions on Event sequence of events and ... events follows specification sequences of events allowed to happen in realtime The execution of the system produces eventsCalvar et al.
    • Runtime monitoring use cases Web applications: filter out messages sent by a browser when the client-server protocol is not followed - Saves CPU time on server (spared handling of erroneous messages) - Useful for developing/debugging the client - The presence of a monitor compels the server to document its protocol in a machine-readable formatCalvar et al.
    • Runtime monitoring use cases An example, the BeepBeep runtime monitor (Hallé and Villemaire) JavaScript Internet XMLHttpRequest Normal situation: JavaScript object XmlHttpRequest centralizes all communications with the outside worldCalvar et al.
    • Runtime monitoring use cases An example, the BeepBeep runtime monitor (Hallé and Villemaire) JavaScript Internet XMLHttpRequest Runtime monitoring: a new class wraps around this object and checks incoming/outgoing messages before passing them to the original XMLHttpRequestCalvar et al.
    • Runtime monitoring use cases In object-oriented programs: enforce valid use of objects methods according to their state start unknown hasnext() == true hasnext() == false next() hasnext() hasnext() more next() none next() error Correct usage of the Java Iterator class (from Jin et al., PLDI 2011)Calvar et al.
    • Runtime monitoring use cases Example: Rosu et al.s Java-MOP full-binding HasNext(Iterator i) { event hasnext after(Iterator i) : call(* Iterator.hasNext()) && target(i) {} event next before(Iterator i) : call(* Iterator.next()) && target(i) {} fsm : unsafe [ start [ next -> unsafe next -> unsafe hasnext -> safe hasnext -> safe ] ] safe [ alias match = unsafe next -> start hasnext -> safe @match { ] System.out.println("next called without hasNext!"); } }Calvar et al.
    • An example: the Bank class A simple (Java) class offering four methods: void open(int act) void close(int act) void withdraw(int act, int amt) boolean isApproved(int act, int amt) Other classes interact with the Bank to perform transactionsCalvar et al.
    • An example: the Bank class As with the Iterator, there are constraints one must follow to use the Bank properly: S1. No operation on an account can be made after close S2. For the same account, the amount cannot change between an approval request and the following call to withdraw Declarative constraints: specify what must be true, but not how to check/enforce itCalvar et al.
    • Issues in runtime monitoring The properties deal with sequences of method calls AND values of the arguments inside these calls S1: close(x) prevents any other method call, but only for that same x S2: requires that methods isApproved(x,y) and the subsequent withdraw(x,y) have the same value y, but only if they also have the same value x These are called data-aware constraints. Why is it problematic?Calvar et al.
    • Data-aware constraints These properties cannot be expressed by a simple finite-state automaton withdraw(x,y) How to express constraint open(x) isApproved(x,y) on x and y? close(x) x=1Calvar et al.
    • Data-aware constraints Existing runtime monitors extend the representation by tracking multiple instances of an automation, parameterized by some value withdraw(x,y) withdraw(x,y) open(x) isApproved(x,y) open(x) isApproved(x,y) close(x) close(x) x=1 x=2Calvar et al.
    • Data-aware constraints If multiples values are involved in the same constraint, one must use nested automata withdraw(x,y) open(x) isApproved isApproved (x,y) withdraw (x,y) withdraw ... (x,y) (x,y) y=1 y=2 x=1Calvar et al.
    • Data-aware constraints Issue #1: the same event may (or may not) need to be dispatched to more than one automaton... ...the specification must hence also describe how to combine the (potentially conflicting) decision returned by each automaton m1(x,y) m1(x,y) m2(y,x) m2(y,x) y=2 y=1 x=1 x=2 ERROR OK and? / or?Calvar et al.
    • Data-aware constraints Issue #2: the specification must describe how and when automata must be instantiated, depending on the events that are observed Consequence: a growing part of the constraints becomes procedural, and amounts to a hand-coding of a monitoring algorithm inside the specificationCalvar et al.
    • Separation of concerns Monitor and specification become tangled: loss of separation of concerns System Monitor Specification } Event Verifies that Gives conditions on Event sequence of events and ... events follows specification sequences of events allowed to happen in realtime The execution of the system produces eventsCalvar et al.
    • Data-aware constraints Issue #3: current monitoring frameworks are very language-dependent; the specification contains code in the systems native languageCalvar et al.
    • Separation of concerns Monitor and system become tangled: loss of separation of concerns (again) System Monitor Specification } Event Verifies that Gives conditions on Event sequence of events and ... events follows specification sequences of events allowed to happen in realtime The execution of the system produces eventsCalvar et al.
    • Monitoring framework Proposed architecture: framework that clearly separates all three aspects of the process - Separation of the system from the events it produces - Separation of the specification from the algorithm to verify itCalvar et al.
    • Monitoring framework Pointcut Event m formatter Monitor Advice Outcome execution Program Aspect Program events are declared using a signature Event data is extracted and formatted as an XML structure XML is passed to a general-purpose monitor Specification is expressed as constraints on sequences of XML structures -> system-independentCalvar et al.
    • Event formatting Example: a call to myBank.withdraw(123, 500) is formatted as <call> Method name <method>withdraw</method> <act>123</act> Method arguments, named as <amt>500</amt> in the methods prototype </call> For a message m, data can be referred to by a path expression π; hence if π = call/method/act, m(π) = "123"Calvar et al.
    • Properties on event sequences LTL-FO+ is an extension of classical Linear Temporal Logic to express constraints on XML event sequences with data - Ground terms are equalities: between a variable and a constant (x = c) between two variables x = y between a path expression and a constant (a/b/c = d)Calvar et al.
    • Properties on event sequences - Boolean connectives carry their traditional meaning - Temporal operators express modalities between messages: G φ : all messages from now on satisfy φ F φ : some message in the future will satisfy φ X φ : the next message will satisfy φ φ U ψ : φ holds on every message until some message satisfies ψCalvar et al.
    • Properties on event sequences - First-order quantifiers fetch and retain values in a message ∀ x ∈ π : φ(x) In the current message m, for every value c such that m(π) = c, φ(c) is true ∃ x ∈ π : φ(x) In the current message m, for some value c such that m(π) = c, φ(c) is trueCalvar et al.
    • Properties on event sequences S1. No operation on an account can be made after close G (call/method = close → ∀ a ∈ call/act : X G (∀ a ∈ call/act : a ≠ a ))Calvar et al.
    • Writing the specification PROTOTYPES void Bank.open(int act); Method calls to void Bank.close(int ac); intercept void Bank.withdraw(int act, int amount); boolean Bank.isApproved(int act, int amt); SPEC LTL-FO+ G ((call/method/close) -> ([a1 call/act] Property to (X (G ([a2 call/act] (!((a1) = (a2)))))))) monitor @FALSE { Action to take System.out.println("This is forbidden"); if property is } violated PROTOTYPES shields the property from language- specific considerationsCalvar et al.
    • Properties on event sequences For Java, a PHP script converts the PROTOTYPES section into the appropriate event formatter, and the @FALSE section into the appropriate advice executor... Pointcut Event m formatter Monitor Advice Outcome execution Program Aspect ...the monitor itself is always the same!Calvar et al.
    • Non-primitive objects What if the methods arguments are not primitive types? Example: Account a = new Account(...); myBank.withdraw(a, 500); <call> <method>withdraw</method> <act> What is the accounts "value"? </act> <amt>500</amt> </call>Calvar et al.
    • Non-primitive objects Observation: runtime constraints are expressed about objects specific properties that ultimately boil down to primitive types Account a = new Account(...); myBank.withdraw(a, 500); The constraint on withdraw only requires account numbers, not full instances of Account Solution: have the event formatter write down member fields of the object (not the full object), but only those necessary for the constraint to monitorCalvar et al.
    • Non-primitive objects Example: PROTOTYPES void AdvancedBank.getAuthorization (Account a, int amount); <return> <method>getAuthorization</method> Message template used <id>{return.getId()}</id> by event formatter for <amount>{return.getAmount()}</amount> this method call </return> void AdvancedBank.withdraw(Account a, Authorization auth); <call> Code to execute on <method>withdraw</method> callee or its arguments <id>{auth.getId()}</id> to populate element <amount>{auth.getAmount()}</amount> with primitive value <call>Calvar et al.
    • Pros of this approach 1. Constraints involving non-primitive objects can be expressed and monitored: once events are formatted, they are undistinguishable from "primitive" calls by the runtime monitor 2. The trace of events can be saved to a (plain) XML file and verified a posteriori; would be impossible if events contained references to raw Java objectsCalvar et al.
    • Pros of this approach 3. The monitor does not deal with native objects from the system: preserves separation of concerns discussed earlier 4. Being allowed to query the internal state of objects sometimes simplifies the property to monitor - Surprising finding - See paper, "autonomous" vs. "query" approach in experiments - Work in progress: systematize this observation ("piggyback runtime monitoring")Calvar et al.
    • Experiments Runtime monitoring framework tested on the "simple" (primitive) and "advanced" (non-primitive) Bank examples, with randomly generated calls 60 Number of traces 50 40 30 20 10 0 [-.08, [-.04,[-.02, [0, [.02,.08) [-.06,-.03) [-.03,0) [0,.02) [.02, [.04,.08) -.04) -.02) 0) .02) .04) S1 S2 Overhead per method call (ms) Simple bankCalvar et al.
    • Conclusion We proposed a framework for the runtime monitoring of events in a system These events can carry data parameters that are relevant to the constraints to monitor Our framework clearly separates the extraction of event data from the system from the property to monitor Experiments show that the formatting of native events into XML imposes an overhead of 5-10 microseconds per eventCalvar et al.