Fast SOA with Apache Synapse
Upcoming SlideShare
Loading in...5
×
 

Like this? Share it with your network

Share

Fast SOA with Apache Synapse

on

  • 18,201 views

Presented at ApacheCon Europe 2008: this presentation introduces the ApacheCon Synapse ESB and the performance benefits of the design.

Presented at ApacheCon Europe 2008: this presentation introduces the ApacheCon Synapse ESB and the performance benefits of the design.

Statistics

Views

Total Views
18,201
Views on SlideShare
18,143
Embed Views
58

Actions

Likes
8
Downloads
398
Comments
1

4 Embeds 58

http://www.slideshare.net 53
http://jisi.dreamblog.jp 2
http://localhost 2
http://translate.googleusercontent.com 1

Accessibility

Categories

Upload Details

Uploaded via as Microsoft PowerPoint

Usage Rights

© All Rights Reserved

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Processing…
  • esper on slide 15
    Are you sure you want to
    Your message goes here
    Processing…
Post Comment
Edit your comment

Fast SOA with Apache Synapse Presentation Transcript

  • 1. High Speed SOA with Apache Synapse Paul Fremantle VP, Apache Synapse CTO, WSO2 [email_address]
  • 2. Contents
    • What is Apache Synapse
    • History of the project
    • Some examples
    • Delving into the model
    • Performance
    • Connectors, Adaptors
    • Getting started
    • Getting involved
  • 3. What is Apache Synapse?
    • Is it?
      • A highly performant XML Gateway
      • An Enterprise Service Bus
      • An HTTP proxy server
      • A protocol switcher
      • A way of hosting message manipulation logic
  • 4. What is Apache Synapse?
    • A lightweight Enterprise Services Bus (ESB)
      • Available as a WAR file, NT Service, Linux Daemon
      • Runs as a process with its own Listeners, Tasks and Senders
      • Can be deployed standalone or part of a cluster or distributed network
      • High performance, asynchronous, streaming design
      • Can initiate work – scheduled tasks
      • Supports multiple transports including HTTP, JMS, TCP, SMTP and (S)FTP
      • Simple to extend
  • 5. Flows JMS SMTP inflow log xslt send outflow send xslt log HTTP 8080 HTTPS 8443 JMS SMTP HTTP 8080 HTTPS 8443
  • 6. Synapse startup
    • Using Bouncy castle JAR for Java 1.5
    • Starting Synapse/Java ...
    • Using SYNAPSE_HOME: C:SYNAPS~1.1in..
    • Using JAVA_HOME: c:jdk
    • Using SYNAPSE_XML: -Dsynapse.xml="C:SYNAPS~1.1in.. epositoryconfsynapse.xml"
    • 2007-11-12 12:16:58,250 [-] [main] INFO ServerManager Using the Axis2 Repository C:SYNAPS~1.1in.. epository
    • 2007-11-12 12:17:01,921 [-] [main] INFO SynapseInitializationModule Initializing Synapse at : Mon Nov 12 12:17:01 GMT 2007
    • 2007-11-12 12:17:01,937 [127.0.0.1-pzfdell] [main] INFO SynapseInitializationModule Loading mediator extensions...
    • 2007-11-12 12:17:01,937 [127.0.0.1-pzfdell] [main] INFO SynapseInitializationModule Initializing the Synapse configuration ...
    • 2007-11-12 12:17:01,968 [127.0.0.1-pzfdell] [main] INFO XMLConfigurationBuilder Generating the Synapse configuration model by parsing the XML configuration
    • (some deleted)
    • 2007-11-12 12:17:04,359 [127.0.0.1-pzfdell] [main] INFO HttpCoreNIOSender HTTP Sender starting
    • 2007-11-12 12:17:04,968 [127.0.0.1-pzfdell] [main] INFO HttpCoreNIOListener HTTPS Listener starting on port : 8443
    • 2007-11-12 12:17:04,968 [127.0.0.1-pzfdell] [main] INFO ServerManager Starting transport https on port 8443
    • 2007-11-12 12:17:05,046 [127.0.0.1-pzfdell] [main] INFO ServerManager Ready for processing
  • 7. History of the project
    • August 2005 Kicked off as an incubator proposal
    • December 2005 Milestone 1 released
    • July 2006 Milestone 2 released
    • December 2006 Synapse 0.90 released
    • January 2007 Graduates from Incubator
    • May 2007 Released 1.0
    • December 2007 Becomes a Top Level Project
    • January 2008 Synapse 1.1.1 released
  • 8. A really simple example
    • Exposing an existing SOAP endpoint as XML/HTTP, XML/JMS
    • <definitions>
    • <proxy name=“example&quot; transports=“http jms&quot;>
    • <target>
    • <endpoint>
    • <wsdl uri=“http://remote.com/service?wsdl“
    • service=“RemoteService&quot;
    • port=“RemotePort&quot;/>
    • </endpoint>
    • <outSequence>
    • <send/>
    • </outSequence>
    • </target>
    • </proxy>
    • </definitions>
  • 9. Some other things you can do “out of the box”
    • Content-based routing
      • Direct messages based on properties or XPath expressions
    • Convert from existing formats (CSV, Fixed records, etc) to and from XML
    • Transform XML
      • Using XSLT or XQuery
    • Validate using XML Schema
    • Update databases and enhance messages with extra data
      • DBLookup and DBReport mediators – any JDBC database
    • Switch between SOAP and non-SOAP, any transport
      • HTTP, HTTPS, SMTP/POP3, XMPP, JMS, TCP, Filesystem, FTP, AMQP
    • Add or handle WS-* support including
      • WS-Security, WS-SecureConversation, WS-Trust, WS-ReliableMessaging
    • Load balancing, failover, throttling
  • 10. Registries and Dynamism Synapse “Registry” HTTP extensible read and cache notify
    • “ Transactional updates”
    • existing flows continue to use old config
    • Also graceful restart
    • switch off transport cleanly
    • restart Synapse when all work is done
  • 11. A more detailed example NY Variable Format records London Existing trade database D2243578XX910 D6233578YYYY9 <FEED> <CAK>2243578</CAK> <AccNo>3289019</AccNo> <CAT>AK</CAT> … </FEED> WS-Security (encryption and digital signature) MQ Validation
  • 12. Overview of the flow
    • <sequence name=“new york”>
    • Flatpack with Variable record config
    • E4X script to reformat XML
    • Send to London
    • </sequence>
    • <sequence name=“london”>
    • DBreport
    • Filter
    • Send to MQ
    • </sequence>
  • 13. Results
    • Replicated a 3 month project is 3 days
      • Included enhancing the Flatpack mediator to support Variable Records
      • Added WS-Security over original scenario
      • 1200 lines of config reduced to less than 300
      • Much more maintainable – clean separation of concerns between departments
  • 14. Google Spreadsheet and CSV Google.com CSV Synapse Http Poller Job CSV to XML Atom
  • 15. Esper
    • Event Stream Processing project
      • http://esper.codehaus.org
    • Allows you to define queries on sets of “events”
      • SELECT AVG(price) FROM OrderEvent.WIN:TIME(30 sec)
    • EsperMediator for Synapse makes it simple to add CEP to your SOA
  • 16. Ganglia, Quartz, Esper Synapse Quartz Scheduler Ganglia gmond telnet 8649 <GANGLIA_XML> GMondPoller Job Esper Mediator SELECT AVG(CPU_USER) FROM GANGLIA urn:gmond Alert notification IM Atom or JMS urn:cpu
  • 17.
    • <definitions xmlns=&quot;http://ws.apache.org/ns/synapse&quot;>
    • <task class=&quot;org.fremantle.gmond.GMonTask“ name=“Gmon”>
    • <trigger interval=&quot;5000&quot;/>
    • <property name=&quot;hostname&quot; value=&quot;localhost&quot;/>
    • <property name=&quot;port&quot; value=&quot;8649&quot;/>
    • <property name=&quot;to&quot; value=&quot;urn:gmond&quot;/>
    • </task>
    • <in>
    • <filter source=&quot;get-property('To')&quot; regex=&quot;urn:gmond&quot;>
    • <log/>
    • <class name=&quot;org.fremantle.esper.EsperMediator&quot;>
    • <property name=&quot;Configuration&quot;
    • value=“./repository/conf/esper.conf.xml&quot;/>
    • <property name=&quot;statement&quot;
    • value=&quot;select avg(cpu_user) from Ganglia.win:length_batch(10) &quot;/>
    • <property name=&quot;EventToAddress&quot; value=&quot;urn:cpu&quot;/>
    • </class>
    • </filter>
    • <filter source=&quot;get-property('To')&quot; regex=&quot;urn:cpu&quot;>
    • <send>
    • <endpoint>
    • <address uri=&quot;http://localhost:9090/notfication&quot;/>
    • </endpoint>
    • </send>
    • </filter>
    • </in>
    • </definitions>
  • 18. Performance
    • Non-blocking IO
      • Apache HTTPCore
      • http://hc.apache.org/httpcomponents-core/httpcore-nio/index.html
    • Streaming XML and Binary
      • Apache Axiom
      • http://ws.apache.org/commons/axiom
  • 19. Non-blocking graphically
    • This model means:
    • Synapse threads never blocked during normal processing
    • Number of sockets open >> number of threads
    TIME Thread2 Incoming req Socket open Thread1 Socket open Request processing Response processing Outgoing resp Outgoing req Incoming resp Synapse
  • 20. Demonstrating Performance
    • Synapse by default runs
      • 2 listener threads
      • 2 sender threads
      • 8 worker threads
    • Added a 100ms thread sleep to the server
    • Ran 250 concurrent clients for 10000 runs
      • Simply would not have run without NIO
    • Also did a simple test comparing:
      • 346 bytes in/ 1,170 bytes out
      • Direct to Axis2: 7.4ms
      • Via Synapse: 8.1ms – diff = 0.710ms !!
  • 21. XSLT performance
  • 22. Extending Synapse
    • Java Mediators
      • Implement a simple Java API
    • Java Commands
      • Pure POJO with mapping into messages using XPath
    • Scripts
      • Using the Apache BSF project you can write scripts in JavaScript, Groovy and Ruby
    • Synapse extensions
      • First class mediators with their own “Domain” XML configuration
      • Simply drop the JAR into the classpath
    • Tasks
      • Repetitive jobs can be run at set intervals
    • Registries
      • Synapse can dynamically load its configuration from a remote registry
    • Transports
      • Create new connectors to remote systems
  • 23. Simple Mediator: CSV->XML
    • public boolean mediate(MessageContext mc) {
      • DataHandler dh = PayloadHelper. getBinaryPayload (mc);
      • BufferedReader br;
      • new BufferedReader(new InputStreamReader(dh.getInputStream()));
      • CSVReader csvReader = new CSVReader(br);
      • OMFactory fac = OMAbstractFactory. getOMFactory ();
      • OMElement el = fac.createOMElement(&quot;csv&quot;, csvNS);
      • // create element to hold data
      • while ((nextLine = csvReader.readNext()) != null) {
      • rownum++;
      • // add elements to XML
      • }
      • br.close();
      • PayloadHelper. setXMLPayload (mc, el);
      • return true;
    • }
  • 24. Tasks – Quartz integration
    • Simple repetitive actions
    • Can also be used to start a long-running activity at startup
    • Uses the Quartz Scheduler to run items
      • www.opensymphony.com/quartz
    • Tasks must implement the Task interface
        • package org.apache.synapse.startup;
        • public interface Task
        • {
        • public abstract void execute();
        • }
    • Tasks may implement the ManagedLifecycle interface
    • Properties are set by injection (String and XML)
  • 25. Sample task - MessageInjector
    • public class MessageInjector implements Task, ManagedLifecycle
    • {
    • public void setTo(String url)
    • { to = url; }
    • public void setMessage(OMElement elem)
    • { message = elem; }
    • public void execute() {
    • MessageContext mc =
    • synapseEnvironment.createMessageContext();
    • mc.setTo(new EndpointReference(to));
    • PayloadHelper.setXMLPayload(mc,
    • message.cloneOMElement());
    • synapseEnvironment.injectMessage(mc);
    • }
    • }
  • 26. Script example <script language=&quot;js&quot;><![CDATA[ var rowset = mc.getPayloadXML() var ns = new Namespace(&quot;http://...&quot;); var d6 = rowset..ns::row.(@recordname==&quot;D6&quot;); var output = <FEEDS/> for (var i=0; i<d6.length(); i++) { var row = d6[i]; var cak = row.ns::entry.(@name==“CAK&quot;).text(); … var feed = <FEED> <CAK>{cak}</CAK> <AccNo>{accno}</AccNo> </FEED>; output.appendChild(feed); } mc.setPayloadXML(output); </script>
  • 27. Community Site for extensions http://esbsite.org
  • 28. Connectors, Adapters, Other projects
    • CSV, Fixed and Variable records – FlatPack, OpenCSV
    • FIX transport using QuickFix/J
    • AMQP transport using Apache QPid
    • Complex Event Processing based on Esper
    • Atom using Apache Abdera
    • Rules using Drools
    • EDI using Smooks
    • BPEL using Apache ODE
    • Hessian Binary XML using Hessian
    • URLRewriter
    • IM Mediator MSN, ICQ, Yahoo, Jabber
  • 29. User quotes
    • &quot;Synapse's simplicity of configuration, ease of extensibility, and integration were the key factors in our choice. Synapse's speed and stability, non-blocking I/O support, and responsive development community then sealed the decision,&quot;
    • &quot;Synapse passed all our performance benchmark, load and memory leak tests with flying colours.&quot;
    • “ Thanks for the quick fix. Synapse continues to rock.”
  • 30. Futures
    • We continue to add new mediators and transports
    • Better support for complete HTTP Proxy
      • Jointly done with HTTPCore
    • More work on a non-XML DSL for Synapse
    • Better support for REST
      • DELETE and PUT support
    • Unifying support for events
      • WS-Eventing, Atom, RSS, XMPP
    • Improved cluster management
      • Graceful restart
    • Improved JMX management
      • Better granularity
  • 31. Getting Started
  • 32. Getting Involved
    • Join us
      • [email_address]
      • [email_address]
    • Raise a JIRA for an improvement you would like
      • http://issues.apache.org/jira/browse/SYNAPSE
    • Submit a patch
    • Contribute a mediator or a task definition
  • 33. Questions?