Global Scale ESB with Mule

  • 2,871 views
Uploaded on

Presentation on Mule 2 and 3 and associated scalability issues and bottlenecks

Presentation on Mule 2 and 3 and associated scalability issues and bottlenecks

More in: Technology
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Be the first to comment
No Downloads

Views

Total Views
2,871
On Slideshare
0
From Embeds
0
Number of Embeds
1

Actions

Shares
Downloads
103
Comments
0
Likes
2

Embeds 0

No embeds

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
    No notes for slide

Transcript

  • 1. Global Scale ESB with Mule Andrew D Kennedy grkvlt@apache.org March 2011
  • 2. Who Am I• Enterprise Java Developer• Apache Qpid Committer• Previously Security Engineer• Over 15 Years Java Experience• Worked with various Investment Banks• Interested in Data Mining and Visualisation
  • 3. This Presentation• What is an ESB• What is Mule• Mule Applications• Mule Scalability
  • 4. What is an ESB• Enterprise Service Bus• SOA – Service Oriented Architecture• SEDA – Staged Event-Driven Architecture• EIP – Enterprise Integration Patterns
  • 5. Service Oriented Architecture• Services – Shared Business Functions• Decoupling – Local and Remote Services• Distributed Applications• Directory and Discovery
  • 6. EIP• Similar to Design Patterns – Gang of Four Book• Shared Terminology• Names for Common Concepts• Allows Discussion• Simplifies Design
  • 7. (Some) Common Patterns• Aggregator (268)• Channel Adapter (127)• Content Based Router (230)• Dead Letter Channel (119)• Selective Consumer (515)• Message Translator (85)• Guaranteed Delivery (122)
  • 8. What is Mule• Open Source Project• MuleSoft – Mule Forge – Mule Enterprise – Mule MQ – Tcat Server• 2.x - Legacy Applications• 3.1.1 - Current Community Release• 3.2.0 - Latest Developer Build
  • 9. Transports• Carry Messages between Services• Connector – Configuration – Threading – Retry• Endpoint – Connects Services – Transform Messages – Transaction Boundary
  • 10. Endpoint Examples<file:inbound-endpoint path="/esb/in" comparator="org.mule.transport.file.comparator.Older FirstComparator" reverseOrder="true” /><jetty:endpoint name="server" host="localhost" port="60203" path="services/Lookup" /><imaps:endpoint name=”mail" host="localhost" password=”hunter2" port="123" user=”adk"/><quartz:endpoint name=”batch" repeatCount="10" repeatInterval=”60000" jobName="job"/><jms:inbound-endpoint queue="test.queue"/>
  • 11. VM• Communication within VM• Synchronous or Asynchronous• Transactional<vm:connector name="async" queueEvents="true” /><vm:inbound-endpoint path="in" connector-ref="async"/>
  • 12. JMS• Queues• Topics• Request-Response<jms:connector name=”jms" specification="1.1" connectionFactory-ref=”qpid" username=”adk" password=”hunter2” /><jms:outbound-endpoint queue=”audit” /><jms:inbound-endpoint topic=”uk.*.gbp” />
  • 13. Web Services• HTTP and HTTPS – Custom Data Formats• CXF – JAX-WS – SOAP – WSDL• REST
  • 14. CXF JAX-WS<inbound-endpoint address="http://localhost:63081/hello" /><cxf:jaxws-service serviceClass="org.example.ServiceImpl” /><cxf:jaxws-client clientClass=”com.example.Client” wsdlPort="SoapPort” wsdlLocation="classpath:/save.wsdl” operation=”output” /><outbound-endpoint address="http://www.example.com/services/save” />
  • 15. File System• Poll Directory for Input<file:connector name="input" fileAge="500" autoDelete="true" pollingFrequency="100" moveToDirectory="/backup" moveToPattern="#[header:originalFilename].backup” />• Write Output File<file:connector name="output" outputAppend="true" outputPattern="#[function:datestamp]- #[header:originalFilename]" />
  • 16. Other Transports• AJAX • TCP• jBPM • UDP• JDBC • FTP• Quartz • POP3• STDIO • IMAP• RMI • SMTP• Servlet • XMPP
  • 17. Transformers• Change message content• XML to POJO via JAXB• Applied to endpoints• Efficient mechanism for processing• Similar to AOP
  • 18. Transformer Types• Encryption and Decryption• ByteArray to Object• Expression Evaluator• XML and JSON• Compression• Encoding and Decoding
  • 19. Filtering• Apply to endpoints• Payload Type• Expression – XPath, OGNL, JXPath• RegEx• Wildcard• Message Property• Logical Operators – And, Or, Not
  • 20. Filtering Examples<and-filter> <message-property-filter pattern="JMSCorrelationID=1234567890"/> <message-property-filter pattern="JMSReplyTo=null"/></and-filter><or-filter> <payload-type-filter expectedType="java.lang.String"/> <payload-type-filter expectedType="java.lang.StringBuffer"/></or-filter><not-filter> <wildcard-filter pattern=”com.example.*"/></not-filter>
  • 21. Routers• Control message flow at endpoints• Used to link services• Both inbound and outbound• Specify Filtering and Transformations
  • 22. Selective Consumer<inbound> <selective-consumer-router> <mulexml:jxpath-filter expression="msg/header/resultcode = success"/> </selective-consumer-router> <forwarding-catch-all-strategy> <jms:endpoint topic="error.topic"/> </forwarding-catch-all-strategy></inbound>
  • 23. Idempotent Filter<inbound> <idempotent-receiver-router idExpression="#[message:id]-#[header:foo]"> <simple-text-file-store directory="./idempotent” /> </idempotent-receiver-router></inbound>
  • 24. Filtering Router<outbound> <forwarding-catch-all-strategy> <jms:outbound-endpoint queue="error.queue"/> </forwarding-catch-all-strategy> <filtering-router> <smtp:outbound-endpoint to=”adk@example.com"/> <payload-type-filter expectedType="java.lang.Exception"/> </filtering-router> <filtering-router> <jms:outbound-endpoint queue="string.queue"/> <and-filter> <payload-type-filter expectedType="java.lang.String"/> <regex-filter pattern="the quick brown (.*)"/> </and-filter> </filtering-router></outbound>
  • 25. Services• Combine Endpoints, Routers and Components• Mule 2.0 idiom• Replaced with Flows and Patterns – Simpler to use and configure – Some features still not available
  • 26. Flows• Mule 3 idiom• Inbound endpoint• Message processors – Chained execution – Components• Outbound endpoint for one-way• Response for request-response
  • 27. Components• Carry out actions• Business logic in a flow• Simple components – Logging – Passthrough – Testing• Spring or POJO• Web services
  • 28. Message Processors• Aggregator and Splitter• Message Filters• Recipient List• Resequencer• Round Robin• Wire Tap
  • 29. Example Flow<flow name=”OrderFlow"> <file:inbound-endpoint path="/incoming"> <file:filename-filter name=”order-*.xml"/> </file:inbound-endpoint> <xml:xslt-transformer xsl-file=”order-transform.xsl"/> <splitter expression="xpath://order"/> <!-- The following message processors will be invoked for each order in the xml file --> <expression-filter expression="xpath://order[@type=book]"/> <component class=”com.example.BookOrderProcessor"/> <smtp:outbound-endpoint subject="Order Confirmation" address="#[variable:email]"/> <jdbc:outbound-endpoint ref=“saveOrder"/ > <default-exception-strategy> <jms:outbound-endpoint queue="failedOrders"/> </default-exception-strategy></flow>
  • 30. JMS Flow<flow name=”JmsFlow"> <jms:inbound-endpoint queue="in"> <jms:transaction action="ALWAYS_BEGIN" /> </jms:inbound-endpoint> <component class="com.example.ProcessMessage" /> <jms:outbound-endpoint queue="out"> <jms:transaction action="ALWAYS_JOIN" /> </jms:outbound-endpoint> <default-exception-strategy> <commit-transaction exception-pattern="com.example.ExampleException” /> <jms:outbound-endpoint queue="dead.letter"> <jms:transaction action="JOIN_IF_POSSIBLE" /> </jms:outbound-endpoint> </default-exception-strategy></flow>
  • 31. Complex Flow<flow name=”TwitterFlow”> <poll frequency="5000"> <twitter:search query=”example" /> </poll> <splitter evaluator="json" expression="results" /> <idempotent-message-filter idExpression="#[json:id]" /> <enricher target="#[variable:userDescription]" source="#[json:description]"> <twitter:user userId="#[json:from_user]" /> </enricher> <salesforce:create type="Opportunity"> <salesforce:sObject> <salesforce:field key="Name" value="#[json:from_user]" /> <salesforce:field key="Description" value="#[variable:userDescription]" /> </salesforce:sObject> </salesforce:create></flow>
  • 32. Patterns• Specific integration Features• Simple Service – Exposes Components as Web Services• Web Service Proxy• Bridge – Message Adapter and Transformer• Validator – Validates inbound messages synchronously – Processes asynchronously
  • 33. Questions?
  • 34. Mule ApplicationMule 2 Application Design
  • 35. Mule Scalability• Global Enterprises• Performance• Multiple Data Locations, Sources, Destinations and Owners• Services – Software as a Service – Infrastructure as a Service – Platform as a Service
  • 36. Problems• Performance – Computation – Latency – Throughput – Infrastructure• Integration – Adding Functionality – Collecting Data• Security and Identity• Transactions
  • 37. Transactions• ACID – Atomicity, Consistency, Isolation, Durability• Easy on one system• Hard with distributed systems – Pick your boundaries• Robust failure handling
  • 38. Computation• Parallel Processing• Asynchronous Tasks• Orchestration – Workflow• Pooling and Threading• More and faster is better
  • 39. Bottlenecks• Bandwidth – Networking – Latency and Throughput• External service providers• Data sources – Database• Messaging
  • 40. Messaging Backbone• Sets scalability limits• Message flow through system• Infrastructure choices – JMS – Web based – AMQP – XMPP – TIBCO• Hardware solutions available
  • 41. Clustering• Various external components – Database – Messaging – Web Services• Business logic components – Terracotta – GigaSpaces XAP
  • 42. Load Balancing• Web Services• Software• Hardware – XML Transformation – SSL• Round Robin
  • 43. Questions?Thanks for ListeningAndrew D Kennedygrkvlt@apache.org