Aceu2009 Apache Synapse Events


Published on

Published in: Technology
  • Be the first to comment

  • Be the first to like this

No Downloads
Total views
On SlideShare
From Embeds
Number of Embeds
Embeds 0
No embeds

No notes for slide
  • Aceu2009 Apache Synapse Events

    1. 1. Event Driven Architectures with Apache Synapse Paul Fremantle VP, Apache Synapse Member, ASF CTO and Co-Founder, WSO2
    2. 2. Why am I talking about this? <ul><li>CTO and Co-Founder, WSO2 </li></ul><ul><ul><li>Previously STSM in IBM WebSphere </li></ul></ul><ul><li>VP, Apache Synapse </li></ul><ul><li>Apache Member </li></ul><ul><li>PMC Member: </li></ul><ul><ul><li>Incubator, WebServices, HTTPComponents, Qpid </li></ul></ul>
    3. 3. Plan of Attack <ul><li>Brief introduction to Apache Synapse </li></ul><ul><li>Understanding SOA approaches </li></ul><ul><li>Event Driven Architecture </li></ul><ul><ul><li>Why would you use it? </li></ul></ul><ul><ul><li>Options and approaches </li></ul></ul><ul><li>How Events fit into Synapse </li></ul><ul><li>Case Study </li></ul><ul><li>Futures </li></ul>
    4. 4. Why use an Event Driven Architecture?
    5. 5. Empowerment
    6. 6. Understanding the point of SOA <ul><li>SOA is fundamentally about empowered distributed systems </li></ul><ul><li>Giving each department/organization/website/etc control of their own systems </li></ul><ul><ul><li>Amazon is a great example of this </li></ul></ul><ul><ul><li>Went from a traditional three tier data/app/web architecture </li></ul></ul><ul><ul><li>Enabled small teams to be effective </li></ul></ul>
    7. 7. Two different kinds of SOA Consumers ESB Services Services Services Consumers Consumers System ESB System System System System System
    8. 8. Wiring <ul><li>In a process-led or mashup-led SOA, the wiring is done using ESB configuration or BPEL </li></ul>
    9. 9. Scenario – Financial Security blocking Database legacy flat file NEW YORK Existing System Synapse Poll Record->XML XML->XML Send LONDON Synapse Split/Iterate DBLookup/Filter Transform to MQ Send Existing System XML/JMS
    10. 10. Overall ESB flow FileSystemListener NY Sequence Log Out-Only Flatpack E4X Send to London LondonEndpoint LondonSequence Iterate DBReport – log whole msg Filter/Drop NOAC Send
    11. 11. PushMePullYou
    12. 12. PushMePullYou scenario
    13. 13. Logical model <ul><li>Every Service Provider and Consumer understands the “canonical XML” </li></ul><ul><ul><li>Of course in real life there will be adapters </li></ul></ul><ul><ul><li>These may be running in an ESB </li></ul></ul><ul><ul><ul><li>But ideally their OWN ESB (or part of a distributed ESB) </li></ul></ul></ul><ul><li>The aim is that each department/group can “own” their own gunk </li></ul><ul><ul><li>Not reliant on central systems to adapt </li></ul></ul>
    14. 14. Event approach <ul><li>Eventing takes this one step further: </li></ul><ul><ul><li>Its up to you to publish to the right place </li></ul></ul><ul><ul><li>Its up to you to subscribe to the right events </li></ul></ul><ul><li>You own the wiring too </li></ul><ul><ul><li>Allows for situational integration </li></ul></ul>
    15. 15. Loose coupled and Self Organizing
    16. 16. We know this!
    17. 17. Event based models
    18. 18. Actuators and Sensors <ul><li>An actuator emits an event </li></ul><ul><li>A sensor accepts events </li></ul>
    19. 19. The Synapse model of Events <ul><li>Loosely based on WS-Eventing </li></ul><ul><ul><li>But can be augmented with other models </li></ul></ul><ul><ul><li>Also more powerful </li></ul></ul><ul><ul><ul><li>Easily add mediation into the flow </li></ul></ul></ul><ul><li>Main concept: </li></ul><ul><ul><li>Event Source </li></ul></ul><ul><ul><ul><li>An Actuator </li></ul></ul></ul>
    20. 20. Advanced Web Services Core Axis © WSO2 Inc. 2006 WS-Eventing Event Source Subscribe with EPR of subscriber Respond with EPR of SubsMgr Subscriber Event Notifications (any message) SubsMgr getStatus renew unsubscribe
    21. 21. Synapse configuration example <ul><li><eventSource name=&quot;SampleEventSource&quot;> </li></ul><ul><li><subscriptionManager class=&quot;o.a.s.eventing.managers.DefaultInMemorySubscriptionManager&quot;> </li></ul><ul><li> <property name=&quot;topicHeaderName&quot; value=&quot;Topic&quot;/> </li></ul><ul><li> <property name=&quot;topicHeaderNS&quot; value=&quot;;/> </subscriptionManager> </li></ul><ul><li></eventSource> </li></ul><ul><li><sequence name=&quot;PublicEventSource&quot; > </li></ul><ul><li><log level=&quot;full&quot;/> </li></ul><ul><li><property name=“OUT_ONLY” value=“true” action=“SET”/> </li></ul><ul><li><eventPublisher eventSourceName=&quot;SampleEventSource&quot;/> </li></ul><ul><li></sequence> </li></ul><ul><li><proxy name=&quot;EventingProxy&quot;> </li></ul><ul><li><target inSequence=&quot;PublicEventSource&quot; /> </li></ul><ul><li></proxy> </li></ul>
    22. 22. Eventing in Synapse Synapse Proxy Event Source Event Publisher Subs Mgr Subscriber Publisher Subscriber Subscriber subscribe Mediation Sequence
    23. 23. Different subscription models <ul><li>WS-Eventing message </li></ul><ul><li>Static subscription in Synapse XML </li></ul><ul><li>Store-dependent </li></ul><ul><ul><li>E.g. Registry model </li></ul></ul>
    24. 24. Completely transport AND WS-Eventing agnostic <ul><li>You can have any kind of message republished to any kind of endpoint </li></ul><ul><ul><li>SMTP -> Jabber </li></ul></ul><ul><ul><li>SOAP -> JMS </li></ul></ul><ul><li>You don’t HAVE to use WS-Eventing to subscribe </li></ul><ul><li>You can bridge into JMS Topics too </li></ul><ul><ul><li>Or bridge from JMS Topics </li></ul></ul>
    25. 25. Registry-based subscription model <ul><li>A header carries the “Topic” </li></ul><ul><ul><li>e.g. /config/hardware/server/windows/xp </li></ul></ul><ul><li>Subscribers can subscribe to a specific topic, or all sub-events </li></ul><ul><li>The topic space is represented as a tree in the registry </li></ul><ul><ul><li>Subscriptions are simply URLs stored as entries at a point in the tree </li></ul></ul><ul><li>/config/ </li></ul><ul><li>/software/ </li></ul><ul><li>/hardware/ </li></ul><ul><li> /server/ </li></ul><ul><li>/linux/ </li></ul><ul><li>/windows </li></ul><ul><li>/xp/ URL1 </li></ul><ul><li>/ URL2 (etc)` </li></ul>
    26. 26. Case Study
    27. 27. Enterprise IT Management <ul><li>Problem statement: </li></ul><ul><ul><li>Customers have multiple installed management systems </li></ul></ul><ul><ul><ul><li>Network Management </li></ul></ul></ul><ul><ul><ul><li>User Management </li></ul></ul></ul><ul><ul><ul><li>Systems Management </li></ul></ul></ul><ul><ul><li>All from the same vendor! </li></ul></ul><ul><ul><li>These are not just “stock” systems – each has been customized for each installation </li></ul></ul><ul><ul><li>Customers have to keep these systems in sync </li></ul></ul><ul><ul><ul><li>By data entry </li></ul></ul></ul><ul><ul><li>Any solution needs to be flexible, extensible, modifiable </li></ul></ul>
    28. 28. A difficult problem <ul><li>Synchronizing multiple different systems </li></ul><ul><li>But: </li></ul><ul><ul><li>Systems have different underlying formats </li></ul></ul><ul><ul><li>Some of the systems may be more accurate than others </li></ul></ul><ul><ul><li>Need to be able to scale to different numbers of systems </li></ul></ul><ul><ul><li>Must be extensible / reprogrammable </li></ul></ul>
    29. 29. Feedback!
    30. 30. Feedback problems Black Box change update
    31. 31. Feedback loops Black Box system Black Box system Adapter Adapter Event Broker
    32. 32. EDA + MDR
    33. 33. What is Complex Event Processing? <ul><li>Simple Event Processing </li></ul><ul><ul><li>Acting on single events </li></ul></ul><ul><ul><li>e.g. a <filter> in the ESB </li></ul></ul><ul><ul><li>Is this a gold or platinum customer? </li></ul></ul><ul><li>Event Stream Processing </li></ul><ul><ul><li>Looking across multiple events </li></ul></ul><ul><ul><li>Finding patterns – e.g. the CPU utilization has been more than 90% for the last 10 minutes </li></ul></ul><ul><li>Complex Event Processing </li></ul><ul><ul><li>Looking across multiple event streams </li></ul></ul><ul><ul><li>e.g There has been a significant increase in overall trading activity AND the average price of commodities has fallen 2% in the last 4 hours </li></ul></ul>
    34. 34. Complex Event Processing in Open Source <ul><li>Esper </li></ul><ul><ul><li> </li></ul></ul><ul><ul><li>A Java library that can be integrated into multiple systems </li></ul></ul><ul><ul><li>A GPLv2 project </li></ul></ul><ul><ul><ul><li>(with a commercial edition available) </li></ul></ul></ul><ul><ul><li>Supports multiple query models </li></ul></ul><ul><ul><ul><li>Based on a SQL-like language </li></ul></ul></ul><ul><ul><ul><li>Grouping, aggregation, sorting, filtering and merging of event streams </li></ul></ul></ul><ul><ul><ul><li>Windows based on time, length, sorted, and others </li></ul></ul></ul><ul><ul><li>Events can be XML, Map, Object </li></ul></ul>
    35. 35. Complex Event Processing examples <ul><li>Event processing languages are typically based on SQL </li></ul><ul><ul><li>With extensions! </li></ul></ul><ul><li>select * from </li></ul><ul><li>select AVG(CPU) FROM sec) </li></ul><ul><li>insert into WarningStream select trainStation, avg(waitTime) as avgWait from MyWindow group by trainStation having avg(waitTime) > 60 </li></ul>
    36. 36. Esper + Synapse <ul><li>Separate project – SciFlex </li></ul><ul><ul><li> </li></ul></ul><ul><li>Allows you to “inject” events from Synapse to be processed by Esper </li></ul><ul><li>Interesting events come back to you </li></ul>
    37. 37. Summary <ul><li>EDA is an interesting approach </li></ul><ul><li>If you design correctly, you can build a really distributed, loosely coupled system </li></ul>
    38. 38. Questions?