Fast SOA with Apache Synapse

High Speed SOA with Apache Synapse Paul Fremantle VP, Apache Synapse CTO, WSO2 [email_address]

Contents
What is Apache Synapse
History of the project
Some examples
Delving into the model
Performance
Connectors, Adaptors
Getting started
Getting involved

What is Apache Synapse?
Is it?
A highly performant XML Gateway
An Enterprise Service Bus
An HTTP proxy server
A protocol switcher
A way of hosting message manipulation logic

What is Apache Synapse?
A lightweight Enterprise Services Bus (ESB)
Available as a WAR file, NT Service, Linux Daemon
Runs as a process with its own Listeners, Tasks and Senders
Can be deployed standalone or part of a cluster or distributed network
High performance, asynchronous, streaming design
Can initiate work – scheduled tasks
Supports multiple transports including HTTP, JMS, TCP, SMTP and (S)FTP
Simple to extend

Flows JMS SMTP inflow log xslt send outflow send xslt log HTTP 8080 HTTPS 8443 JMS SMTP HTTP 8080 HTTPS 8443

Synapse startup
Using Bouncy castle JAR for Java 1.5
Starting Synapse/Java ...
Using SYNAPSE_HOME: C:SYNAPS~1.1in..
Using JAVA_HOME: c:jdk
Using SYNAPSE_XML: -Dsynapse.xml="C:SYNAPS~1.1in.. epositoryconfsynapse.xml"
2007-11-12 12:16:58,250 [-] [main] INFO ServerManager Using the Axis2 Repository C:SYNAPS~1.1in.. epository
2007-11-12 12:17:01,921 [-] [main] INFO SynapseInitializationModule Initializing Synapse at : Mon Nov 12 12:17:01 GMT 2007
2007-11-12 12:17:01,937 [127.0.0.1-pzfdell] [main] INFO SynapseInitializationModule Loading mediator extensions...
2007-11-12 12:17:01,937 [127.0.0.1-pzfdell] [main] INFO SynapseInitializationModule Initializing the Synapse configuration ...
2007-11-12 12:17:01,968 [127.0.0.1-pzfdell] [main] INFO XMLConfigurationBuilder Generating the Synapse configuration model by parsing the XML configuration
(some deleted)
2007-11-12 12:17:04,359 [127.0.0.1-pzfdell] [main] INFO HttpCoreNIOSender HTTP Sender starting
2007-11-12 12:17:04,968 [127.0.0.1-pzfdell] [main] INFO HttpCoreNIOListener HTTPS Listener starting on port : 8443
2007-11-12 12:17:04,968 [127.0.0.1-pzfdell] [main] INFO ServerManager Starting transport https on port 8443
2007-11-12 12:17:05,046 [127.0.0.1-pzfdell] [main] INFO ServerManager Ready for processing

History of the project
August 2005 Kicked off as an incubator proposal
December 2005 Milestone 1 released
July 2006 Milestone 2 released
December 2006 Synapse 0.90 released
January 2007 Graduates from Incubator
May 2007 Released 1.0
December 2007 Becomes a Top Level Project
January 2008 Synapse 1.1.1 released

A really simple example
Exposing an existing SOAP endpoint as XML/HTTP, XML/JMS
<definitions>
<proxy name=“example" transports=“http jms">
<target>
<endpoint>
<wsdl uri=“http://remote.com/service?wsdl“
service=“RemoteService"
port=“RemotePort"/>
</endpoint>
<outSequence>
<send/>
</outSequence>
</target>
</proxy>
</definitions>

Some other things you can do “out of the box”
Content-based routing
Direct messages based on properties or XPath expressions
Convert from existing formats (CSV, Fixed records, etc) to and from XML
Transform XML
Using XSLT or XQuery
Validate using XML Schema
Update databases and enhance messages with extra data
DBLookup and DBReport mediators – any JDBC database
Switch between SOAP and non-SOAP, any transport
HTTP, HTTPS, SMTP/POP3, XMPP, JMS, TCP, Filesystem, FTP, AMQP
Add or handle WS-* support including
WS-Security, WS-SecureConversation, WS-Trust, WS-ReliableMessaging
Load balancing, failover, throttling

Registries and Dynamism Synapse “Registry” HTTP extensible read and cache notify
“ Transactional updates”
existing flows continue to use old config
Also graceful restart
switch off transport cleanly
restart Synapse when all work is done

A more detailed example NY Variable Format records London Existing trade database D2243578XX910 D6233578YYYY9 <FEED> <CAK>2243578</CAK> <AccNo>3289019</AccNo> <CAT>AK</CAT> … </FEED> WS-Security (encryption and digital signature) MQ Validation

Overview of the flow
<sequence name=“new york”>
Flatpack with Variable record config
E4X script to reformat XML
Send to London
</sequence>
<sequence name=“london”>
DBreport
Filter
Send to MQ
</sequence>

Results
Replicated a 3 month project is 3 days
Included enhancing the Flatpack mediator to support Variable Records
Added WS-Security over original scenario
1200 lines of config reduced to less than 300
Much more maintainable – clean separation of concerns between departments

Google Spreadsheet and CSV Google.com CSV Synapse Http Poller Job CSV to XML Atom

Esper
Event Stream Processing project
http://esper.codehaus.org
Allows you to define queries on sets of “events”
SELECT AVG(price) FROM OrderEvent.WIN:TIME(30 sec)
EsperMediator for Synapse makes it simple to add CEP to your SOA

Ganglia, Quartz, Esper Synapse Quartz Scheduler Ganglia gmond telnet 8649 <GANGLIA_XML> GMondPoller Job Esper Mediator SELECT AVG(CPU_USER) FROM GANGLIA urn:gmond Alert notification IM Atom or JMS urn:cpu

Performance
Non-blocking IO
Apache HTTPCore
http://hc.apache.org/httpcomponents-core/httpcore-nio/index.html
Streaming XML and Binary
Apache Axiom
http://ws.apache.org/commons/axiom

Non-blocking graphically
This model means:
Synapse threads never blocked during normal processing
Number of sockets open >> number of threads
TIME Thread2 Incoming req Socket open Thread1 Socket open Request processing Response processing Outgoing resp Outgoing req Incoming resp Synapse

Demonstrating Performance
Synapse by default runs
2 listener threads
2 sender threads
8 worker threads
Added a 100ms thread sleep to the server
Ran 250 concurrent clients for 10000 runs
Simply would not have run without NIO
Also did a simple test comparing:
346 bytes in/ 1,170 bytes out
Direct to Axis2: 7.4ms
Via Synapse: 8.1ms – diff = 0.710ms !!

XSLT performance

Extending Synapse
Java Mediators
Implement a simple Java API
Java Commands
Pure POJO with mapping into messages using XPath
Scripts
Using the Apache BSF project you can write scripts in JavaScript, Groovy and Ruby
Synapse extensions
First class mediators with their own “Domain” XML configuration
Simply drop the JAR into the classpath
Tasks
Repetitive jobs can be run at set intervals
Registries
Synapse can dynamically load its configuration from a remote registry
Transports
Create new connectors to remote systems

Simple Mediator: CSV->XML
public boolean mediate(MessageContext mc) {
DataHandler dh = PayloadHelper. getBinaryPayload (mc);
BufferedReader br;
new BufferedReader(new InputStreamReader(dh.getInputStream()));
CSVReader csvReader = new CSVReader(br);
OMFactory fac = OMAbstractFactory. getOMFactory ();
OMElement el = fac.createOMElement("csv", csvNS);
// create element to hold data
while ((nextLine = csvReader.readNext()) != null) {
rownum++;
// add elements to XML
}
br.close();
PayloadHelper. setXMLPayload (mc, el);
return true;
}

Tasks – Quartz integration
Simple repetitive actions
Can also be used to start a long-running activity at startup
Uses the Quartz Scheduler to run items
www.opensymphony.com/quartz
Tasks must implement the Task interface
package org.apache.synapse.startup;
public interface Task
{
public abstract void execute();
}
Tasks may implement the ManagedLifecycle interface
Properties are set by injection (String and XML)

Sample task - MessageInjector
public class MessageInjector implements Task, ManagedLifecycle
{
public void setTo(String url)
{ to = url; }
public void setMessage(OMElement elem)
{ message = elem; }
public void execute() {
MessageContext mc =
synapseEnvironment.createMessageContext();
mc.setTo(new EndpointReference(to));
PayloadHelper.setXMLPayload(mc,
message.cloneOMElement());
synapseEnvironment.injectMessage(mc);
}
}

Script example <script language="js"><![CDATA[ var rowset = mc.getPayloadXML() var ns = new Namespace("http://..."); var d6 = rowset..ns::row.(@recordname=="D6"); var output = <FEEDS/> for (var i=0; i<d6.length(); i++) { var row = d6[i]; var cak = row.ns::entry.(@name==“CAK").text(); … var feed = <FEED> <CAK>{cak}</CAK> <AccNo>{accno}</AccNo> </FEED>; output.appendChild(feed); } mc.setPayloadXML(output); </script>

Community Site for extensions http://esbsite.org

Connectors, Adapters, Other projects
CSV, Fixed and Variable records – FlatPack, OpenCSV
FIX transport using QuickFix/J
AMQP transport using Apache QPid
Complex Event Processing based on Esper
Atom using Apache Abdera
Rules using Drools
EDI using Smooks
BPEL using Apache ODE
Hessian Binary XML using Hessian
URLRewriter
IM Mediator MSN, ICQ, Yahoo, Jabber

User quotes
"Synapse's simplicity of configuration, ease of extensibility, and integration were the key factors in our choice. Synapse's speed and stability, non-blocking I/O support, and responsive development community then sealed the decision,"
"Synapse passed all our performance benchmark, load and memory leak tests with flying colours."
“ Thanks for the quick fix. Synapse continues to rock.”

Futures
We continue to add new mediators and transports
Better support for complete HTTP Proxy
Jointly done with HTTPCore
More work on a non-XML DSL for Synapse
Better support for REST
DELETE and PUT support
Unifying support for events
WS-Eventing, Atom, RSS, XMPP
Improved cluster management
Graceful restart
Improved JMX management
Better granularity

Getting Started

Getting Involved
Join us
[email_address]
[email_address]
Raise a JIRA for an improvement you would like
http://issues.apache.org/jira/browse/SYNAPSE
Submit a patch
Contribute a mediator or a task definition

Questions?

http://www.slideshare.net	53
http://jisi.dreamblog.jp	2
http://localhost	2
http://translate.googleusercontent.com	1

Fast SOA with Apache Synapse

by Paul Fremantle on Apr 14, 2008

Accessibility

Categories

Upload Details

Usage Rights

4 Embeds 58

Statistics

Fast SOA with Apache Synapse Presentation Transcript