A Workhorse Named Mule

4,826 views

Published on

Slides from my talk at VIJUG in July 2009.

Published in: Technology, Education
1 Comment
15 Likes
Statistics
Notes
  • Note that the XML configuration part this presentation applies to Mule 2.x. Mule 3 has a more powerful and simplified approach called 'flow', which has replaced the 'service' described in these slides.
       Reply 
    Are you sure you want to  Yes  No
    Your message goes here
No Downloads
Views
Total views
4,826
On SlideShare
0
From Embeds
0
Number of Embeds
101
Actions
Shares
0
Downloads
33
Comments
1
Likes
15
Embeds 0
No embeds

No notes for slide

A Workhorse Named Mule

  1. 1. A workhorse named Mule    
  2. 2. The speaker  Mule in Action, co­author  Project despot of Mule JCR Transport  Common Retry Policies  HTML Dashboard  Mule Community Committee Member  Lead of NxBRE  Judge for the Jolt Awards  And a few other stuff...    
  3. 3. Non­agenda  eMule  Mule pre 2.2.0, EE  Tutorial  Works on my machine  Alternative configurations    
  4. 4. Agenda  Introduction  Exploration  Construction  In Action    
  5. 5. Introduction    
  6. 6. What is Mule? ”Mule is a lightweight messaging framework and  highly distributable object broker”    
  7. 7. What? No ESB? No SOA?  ESB is a deployment topology  Mule can be deployed as ESB  But also: ESN, Hub and spoke...  SOA can leverage an ESB  Just an extra tool in a tool box  Mule has no dogma nor methodology  More on this later...    
  8. 8. And what about Java?  Written in Java  Leverages zillions of Java open source  libraries  Can do a lot without Java though  XML and Groovy can go a long way    
  9. 9. Genesis  Founded by Ross Mason  SourceForge project created in April 2003  Version 1.0 in April 2005  Moved to CodeHaus  Architecture based on:  EIP  SEDA    
  10. 10. EIP  Enterprise Integration Patterns  Hohpe & Woolf (2004)  Gave Mule its lingo  Endpoints, Transformers...  Routers, Bridge...  Gives us patterns & iconography   http://www.enterpriseintegrationpatterns.com  
  11. 11. SEDA  Staged Event­Driven Architecture  Welsh (2001)  Disciplined approach to overload management  Event­driven application  Stages connected by queues  Graceful degradation under load  By default Mule is asynchronous and SEDA    
  12. 12. What's in a name? ”After  working  on  a  couple  of  bespoke  ESB  systems,  I  found  that  there  was  a  lot  of  infrastructure  work  to  be  done  before  you  can  really start thinking about implementing any logic.  I regard this infrastructure work as 'donkey work' as  it  needs  doing  for  every  project.  I  preferred  Mule over Donkey and Ass just didn't seem right. A  Mule  is  also  commonly  referred  to  as  a  carrier  of load, moving it form one place to another. The  load  we  specialise  in  moving  is  your  enterprise  information.”     Ross Mason
  13. 13. Donkey work Integration App 1 Broker App 2 App 3    
  14. 14. Mule work Adaptation, Transformation & Routing App 1 Mule App 2 App 3    
  15. 15. Mule's dual nature    
  16. 16. Mule's dual nature ”Mule is a lightweight messaging framework and  highly distributable object broker”    
  17. 17. Lightweight messaging  framework Mule Your Application    
  18. 18. Highly distributable object broker Mule Mule    
  19. 19. Dual competition  Integration  frameworks  Camel ✔ Strictly EIP  Spring Integration ✔ DI oriented  ESBs  Apache, WSO2,  ✔ Web service  OW, JBoss oriented  Big vendors ✔ Tooling ✔ Doodleware    
  20. 20. Dual offer  Community Edition  Stable “enough”  Upgrades can be rocky  Enterprise Edition  Supported, stabler  Enterprise features  MQSeries connectors, retry policies  Monitoring tool (MuleHQ)    
  21. 21. Key benefits I  Proven product  Used in many world­class applications  Sound architecture  Drastically refined in Mule 2  Numerous protocols and extensions  From MuleSource and MuleForge  Spring & Maven oriented    
  22. 22. Key benefits II  Good tooling  Mule IDE  Galaxy  Active community  Support, MuleForge  Shared code base CE / EE  Professional support available    
  23. 23. Main drawbacks  Unified messaging abstraction  Acts as a potential keyhole on the real deal  Brings its own idiosyncrasies  Like any framework...  Standards based but not standards compliant  But what standards? JBI? SCA?  Not fully EIP but uses much of its lingo    
  24. 24. Exploration    
  25. 25. ”A messaging framework”  Mule is about abstraction and adaptation  Message  Protocol  As a framework  You can code to it  Wisely  You can extend almost every aspect of it  Wisely    
  26. 26. Message abstraction MuleMessage Payload Properties Attachment Exception    
  27. 27. Message abstraction  Payload  HTTP body, File content  Properties  HTTP Headers, JMS properties  Attachments  SOAP parts, Email parts  Exception  Related to the current message processing    
  28. 28. Protocol abstraction  Handled by ”transports”  Connectors and endpoints  Message adapters and transformers  Inbound interactions  Receive (event)  Request (polling)  Outbound interactions  Send (request/response)  Dispatch (asynchronous)    
  29. 29. Protocol adaptation  At connector level  Handles interactions   At message level  Message adaptation  Raw data access  Specific additional transformation  e.g. Object ↔ JMS Message transformers    
  30. 30. Protocol adaptation Inbound Protocol Adaptation Mule Transformation Mule Specific Message Message    
  31. 31. Protocol adaptation Outbound Mule Transformation Mule Adaptation Protocol Message Message Specific    
  32. 32. Unified payloads?  File transport  Stream  java.io.File  JMS transport  Map, String...  IMAP transport  String    
  33. 33. xmlns:context="http://www.springframework.org/schema/context" xmlns:file="http://www.mulesource.org/schema/mule/file/2.2" xmlns:jms="http://www.mulesource.org/schema/mule/jms/2.2" xmlns:xm="http://www.mulesource.org/schema/mule/xml/2.2" xsi:schemaLocation=" http://www.springframework.org/schema/beans http://www.springframework.org/schema/beans/spring-beans-2.5.xsd http://www.springframework.org/schema/context http://www.springframework.org/schema/context/ spring-context-2.5.xsd http://www.mulesource.org/schema/mule/core/2.2 http://www.mulesource.org/schema/mule/core/2.2/mule.xsd http://www.mulesource.org/schema/mule/file/2.2 http://www.mulesource.org/schema/mule/file/2.2/mule-file.xsd http://www.mulesource.org/schema/mule/jms/2.2 http://www.mulesource.org/schema/mule/jms/2.2/ mule-jms.xsd http://www.mulesource.org/schema/mule/xml/2.2 http://www.mulesource.org/schema/mule/xml/2.2/ mule-xml.xsd" > <notifications dynamic="true"> Need help! <notification event="ENDPOINT-MESSAGE"/> </notifications> <global-property name="jms.url" value="vm://localhost"/> <jms:activemq-connector name="jmsConnector" specification="1.1" brokerURL="${jms.url}"/> <file:connector name="FileConnector" streaming="false" autoDelete="true" pollingFrequency="1000"> <file:expression-filename-parser/> </file:connector> <model name="jmsInboundModel"> <service name="jmsInboundService"> <inbound> <jms:inbound-endpoint topic="backup-reports" > <transformers> <jms:jmsmessage-to-object-transformer/> </transformers> </jms:inbound-endpoint>   </inbound>  
  34. 34. Mule's anatomy    
  35. 35. Mule's anatomy Service Service Mule Service Service    
  36. 36. Mule service  Wider definition than strict SOA  Both inbound and outbound are considered  Service can have business logic or not  Message routing is done by services  Role defined by messaging style  Four main styles    
  37. 37. Request Response Request Client Service Response    
  38. 38. Asynchronous Dispatch Client Service    
  39. 39. Synchronous Request Request Client Service Service Response Response    
  40. 40. Async Request Response Request Dispatch Client Service Service Response Response    
  41. 41. Combined styles Request Dispatch Client Service Service Response    
  42. 42. Other inbound styles Listen Source Service Poll Source Service Request Source Service    
  43. 43. Mule's service anatomy Service ?    
  44. 44. Mule's service anatomy Inbound Outbound Routing Component Routing Response    
  45. 45. Mule's service routing anatomy Inbound Outbound Transformers Transformers Endpoints Endpoints Router Router Component Response    
  46. 46. Endpoints  Communication ”touch points”  Configured  Locally in services  Inbound and outbound  Globally  Named  “Unbound”  Used locally in or out    
  47. 47. Endpoints  Inbound <jms:inbound-endpoint topic="events" />  Outbound <file:outbound-endpoint path="./events" />  Global <jms:endpoint name="events.in" topic="events" /> ... <inbound-endpoint ref="events.in" />    
  48. 48. Endpoints URI  Two equivalent endpoints: <jms:inbound-endpoint topic="events" /> <inbound-endpoint address="jms://topic/events" />  Prefer rich XML schema style  Generic style exists for extensibility  In code, URI style prevails    
  49. 49. Routers  Routing behavior for endpoints  Smart “switching” and filtering  Aggregation, multicasting...  Configured locally in services  Different families  Inbound  Outbound  Async­reply    
  50. 50. Routers <inbound> <jms:inbound-endpoint topic="events" /> <jdbc:inbound-endpoint queryKey="eventsQuery"/> </inbound> <outbound> <multicasting-router> <jms:outbound-endpoint topic="events"/> <file:outbound-endpoint path="./events" /> </multicasting-router> <outbound>    
  51. 51. Transformers  Message payload  Generic  Bytes to string, compression, XSLT...  Transport  JMS to object, String to email...  Message properties  CRUD  Round trip alter egos    
  52. 52. Transformers  Configured  Locally on endpoints  In / response and out  Globally  Named  Used locally in or response  Sometimes invisible (transport)  Can be composed  Rolling your own is rare    
  53. 53. Transformers <xm:xslt-transformer name="DocbookToFO" xsl-file="docbook-fo.xsl" /> ... <inbound-endpoint ref="docs.in.sync" > <transformers> <gzip-uncompress-transformer /> <transformer ref="DocbookToFO" /> </transformers> <response-transformers> <gzip-compress-transformer /> </response-transformers> </inbound-endpoint>    
  54. 54. Components  Messaging behavior  Bridge, Log  Reflection message builder  Remote service proxy  Business behavior  Instantiation by Mule or Spring  Pure POJO or Mule aware  Can be pooled    
  55. 55. Components <log-component /> <test:component/> <component> <singleton-object class="com.acme.Component" /> </component> <component> <spring-object bean="yourBean" /> </component> <pooled-component class="com.acme.Component" />    
  56. 56. http://www.springframework.org/schema/beans/spring-beans-2.5.xsd http://www.springframework.org/schema/context http://www.springframework.org/schema/context/ spring-context-2.5.xsd http://www.mulesource.org/schema/mule/core/2.2 http://www.mulesource.org/schema/mule/core/2.2/mule.xsd http://www.mulesource.org/schema/mule/file/2.2 http://www.mulesource.org/schema/mule/file/2.2/mule-file.xsd http://www.mulesource.org/schema/mule/jms/2.2 http://www.mulesource.org/schema/mule/jms/2.2/ mule-jms.xsd http://www.mulesource.org/schema/mule/xml/2.2 http://www.mulesource.org/schema/mule/xml/2.2/ mule-xml.xsd" > <notifications dynamic="true"> <notification event="ENDPOINT-MESSAGE"/> </notifications> Pwned! <global-property name="jms.url" value="vm://localhost"/> <jms:activemq-connector name="jmsConnector" specification="1.1" brokerURL="${jms.url}"/> <file:connector name="FileConnector" streaming="false" autoDelete="true" pollingFrequency="1000"> <file:expression-filename-parser/> </file:connector> <model name="jmsInboundModel"> <service name="jmsInboundService"> <inbound> <jms:inbound-endpoint topic="backup-reports" > <transformers> <jms:jmsmessage-to-object-transformer/> </transformers> </jms:inbound-endpoint> </inbound> <outbound> <pass-through-router> <file:outbound-endpoint path="./data/reports" outputPattern="BACKUP-REPORT-#[function:dateStamp]"/>   </pass-through-router>  
  57. 57. Service configuration <service name="eventsWriter"> <inbound> <jms:inbound-endpoint topic="events" /> </inbound> <outbound> <pass-through-router> <file:outbound-endpoint path="./events" /> </pass-through-router> </outbound> </service>    
  58. 58. Service configuration <service name="eventsWriter"> <inbound> <jms:inbound-endpoint topic="events" /> </inbound> <bridge-component /> <outbound> <pass-through-router> <file:outbound-endpoint path="./events" /> </pass-through-router> </outbound> </service>    
  59. 59. Anything else?  Connectors  Transport specific configuration  Model  Shared configuration & behavior for services  Interceptors  To add cross­cutting concerns around  components    
  60. 60. The mysterious Mule client    
  61. 61. The mysterious Mule client  Why a client?  In­memory access to an instance  Message generation  Functional testing  Remote access to an instance  Proprietary remote invoker  Raw transport usage    
  62. 62. Mule client interactions (some)  Dispatch muleClient.dispatch("jms://queue", map, null);  Send MuleMessage response = muleClient.send("vm://channel", payload, props);  Request MuleMessage requested = muleClient.request("file://events/01.dat", 0);    
  63. 63. Construction    
  64. 64. What do we build?  Standalone  Embedded  Project structure  Java application  NetBoot  Web application  Repository artifacts  EJB application  JCA resource    
  65. 65. Just ship it!  A Mule project is just a bunch of XML files  Tinker with them a little  Ship them to production  And you are good to go!  Right?    
  66. 66.    
  67. 67. No escape from SD laws!  Practices  Middleware development  TDD  Repeatable builds  Repeatable deployments  Tools  SCM  Continuous integration    
  68. 68. Middleware development  practices  Do not get coupled with the middleware  Contain Mule related code to Mule aware  classes  Transformers, interceptors  Caching  Time­outs  Efficient logging and manageability    
  69. 69. Unit Testing  For components, transformers...  Outside of Mule  xUnit    
  70. 70. Functional Testing  For routing, configuration...  Test­scoped instance  Mule client  Test component  Substitution endpoints  In­memory infrastructure  Jitr    
  71. 71. Integration Testing  For Mule instance in situation  Complex because heterogeneous  Test messages  Test services  Secondary endpoints  soapUI, HermesJMS  Test Driven EAI  Hohpe & Istvanik (2002)     http://bit.ly/tdeai
  72. 72. Other Testing  Load tests  Use JMeter  Profiling  Use the YourKit Profiler Pack    
  73. 73. Asynchronous testing challenge    
  74. 74. Asynchronous testing (lucky) Test Thread Mule Thread Start Mule Send test message Process message Perform assertions Stop Mule    
  75. 75. Asynchronous testing Test Thread Mule Thread Start Mule Send test message Perform assertions Process message Stop Mule    
  76. 76. Asynchronous testing Fixed Test Thread Mule Thread Start Mule Send test message Process message Await latch (n) Latch down Perform assertions Stop Mule     http://bit.ly/tasync
  77. 77. Unleashing Mule    
  78. 78. Unleashing Mule I  Generic knowledge  Maven  Spring    
  79. 79. Maven  Mule is available as a set of artifacts  Core, transports, modules  With petazillions of dependencies  Distribution packs them all  Specific archetypes  New project  New transport  Galaxy publication    
  80. 80. Spring  Code is independent from Spring  Configuration is fully Spring XML  With specific schemas  Use Spring configuration practices  DI, properties, multiple files...  Use Spring goodness  Specific elements,  MBean exporter...  Mule's context can use a Spring parent  Bean reuse    
  81. 81. Unleashing Mule II  Specific knowledge  VM Transport  Transport configuration  Expressions  Scripting  Entry point resolvers  Notifications    
  82. 82. VM Transport  In­memory named pipe  Direct or queuing  Can be disk persistent  What you can not solve with one service  You solve it with two services  And a VM Queue in between!    
  83. 83. Transport configuration  Know the underlying transport  Understand what Mule hides & controls  Learn the configuration options  Including alternative adapters, transformers  Leverage the transport­specific properties  Inbound and outbound  Read the source code    
  84. 84. Expressions  Dynamic properties in configuration  #[evaluator:expression]  Filters, transformers, endpoint URIs  Usually evaluated on the current message  Standalone and Mule­wide evaluators exist  Many evaluators available  Groovy, XPath, Map Payload, OGNL...  Can also be invoked from code    
  85. 85. <message-properties-transformer name="SetTo"> <add-message-property key="toAddresses" value="#[xpath:/Receipts/SendToEmail]" /> </message-properties-transformer> ... <smtp:outbound-endpoint host="${smtp.host}" port="${smtp.port}" from="${smtp.fromAddress}" subject="${smtp.subject}" transformer-refs="SetTo StringToEmailMessage" />    
  86. 86. Scripting  Any JSR­223 engine  Groovy bundled with the scripting module  Rhino bundled with JDK 1.6  Transformers, components, expressions  Good for non­generic ones  Concise syntax (i.e. non­Java)  Inlined or standalone scripts  Spring Scripted Beans are also an option    
  87. 87. <scripting:transformer name="toUpperCase"> <scripting:script engine="groovy"> return payload.toUpperCase() </scripting:script> </scripting:transformer> <scripting:component> <scripting:script engine="js"> if (payload.search("STATUS: CRITICAL") != -1) { message.setProperty("PRIORITY", 'HIGH') } return message </scripting:script> </scripting:component> <expression-filter evaluator="groovy" expression="message.payloadAsBytes.length>1024" />    
  88. 88. Entry point resolvers  Secret behind Mule­free components  Locate the method to call  Several resolvers available  Reflection, array, no argument, method name  By default, Mule uses a set of resolvers  Add a transformation in front to better resolve  By type or directly by name    
  89. 89. Notifications  Internal notification framework  Extreme diversity of events  Mule lifecycle, routing, errors, time­outs...  Standard consumers  JMX, Log4J, publish to endpoint  Custom consumers  Components, interceptors...    
  90. 90. In Action    
  91. 91. The tough questions    
  92. 92. The tough questions  SOAP or REST?  Monitoring  Transactions  Auditing  Exception handling  High availability  Security  Hot reload  Scheduling  Governance  Orchestration  Performance    
  93. 93. SOAP or REST?  SOAP transports  REST transports  CXF  Abdera  Axis  Jersey  Restlet    
  94. 94. Transactions  Supported by a handful of transports  VM, JDBC, JMS  Mostly useful for bridging  Like Queue to Topic  Consider eventual consistency  Conversation based  Idempotent services  Compensation    
  95. 95. Exception handling  Exceptions are handled at several levels  Connector and service  Retry policies  Different strategies  Logging, dispatch to endpoint  Transaction roll­back  Exception based outbound router  Localized error handling via fallback    
  96. 96. Security  Secure endpoints  Authentication and authorization  Secure payload  Encryption  Security manager  Can delegate to Spring Security  Can use JAAS  Transport security handled at connector level    
  97. 97. Scheduling  Based on Quartz  In inbound router  To regularly send a fix payload  To request an endpoint (polling)  In outbound router   To schedule a delayed dispatch    
  98. 98. Orchestration  Trivial, short lived  Message flow is the orchestration  Component binding, chaining router  Complex, long lived  Messages trigger worflow steps  jBPM (native)  Orchestrator triggers services  ActiveBPEL (via WS)    
  99. 99. Chaining router <service name="WeatherDispatchService"> <inbound> <inbound-endpoint ref="WeatherRequest" /> </inbound> <outbound> <chaining-router> <outbound-endpoint ref="WeatherChannel" /> <jms:outbound-endpoint queue="weather" /> </chaining-router> </outbound> </service>    
  100. 100. Monitoring  Standard Java middleware  Monitor JVM with SNMP or JMX  Monitor Mule with JMX  Monitor external channels  Queues filling up, DB not refreshed...  EE relies on MuleHQ  Based on Hyperic    
  101. 101. Auditing  Hard with distributed heterogeneous systems  Watermark messages with correlation ID  Standard with JMS  Ad hoc with other transports  Always log correlation ID with all messages  Mule takes great care of correlation ID    
  102. 102. High availability  Easily achieved with stateless instances  Redundancy & network load balancer  Clustering needed with stateful instances  EE will offer GigaSpaces  Terracotta  DB, in some cases  ID store for idempotent receiver    
  103. 103. Mythical hot reload  Everybody wants it  Really?  Mule configuration can be hot reloaded  Since 2.2.2  Mule libraries are OSGi bundles  But Mule is not (yet) running on OSGi    
  104. 104. Governance  Publicly exposed services are commitments  Required by compliance rules  Galaxy  Repository and registry  Versioning  Backwards compatibility check  Galaxy EE is HA    
  105. 105. Governance EE    
  106. 106. Performance    
  107. 107. Performance I  Beware benchmarks  SEDA is tricky to tune  Many thread pools  Some not directly handled by Mule  Threading   Configured with profiles  Affected by transactions, synchrony  Follow MuleSource's methodology    
  108. 108. Performance II  Consider streaming  Smooks for efficient processing  Tune the transports  HTTP chunking, JMS acking...  Profile first!  YourKit Profiler Pack    
  109. 109. The big picture    
  110. 110.    
  111. 111. To ESB or not? I  ESB in the middle of all service calls  Looks good on slides  But a big mess inside  Remember Jim Webber  “Guerilla SOA”  ESB == “Egregious Spaghetti Boxes”     http://bit.ly/g­soa
  112. 112. To ESB or not? II  “A good ESB is a dead ESB”?  Decision points: http://bit.ly/to­esb­or­not  “MuleSource is Open for Integration.”  Don't get confused on branding/naming  ESB is a topology not a tool  There are others...    
  113. 113. Topology  ESB  JMS backbone  ESN  Direct service to service  Hub & Spoke  Centralized broker  Client / Server  ...    
  114. 114. Methodology  None officially preached by MuleSource  But lots of experience with many clients  Fits with the one in­house  Nespresso Open Architecture (NesOA)  Read MuleSource's articles  To get the gist of their approach  Pragmatism first!    
  115. 115. Bottom­up SOA ” I’ve seen firsthand that the more successful enterprises that have adopted SOA have done so at the grassroots level. (...) Certainly many of our customers choose Mule not on a strategic initiative, but because they have pockets of successful implementations and decide to work with us to help pull the pieces together.” http://bit.ly/bup­soa     Ross Mason
  116. 116. Time to hit it!    
  117. 117. Getting started  www.mulesource.org  Download CE, browser user guide  www.mulesource.com  Professional support  http://code.google.com/p/muleinaction/  Mule in Action source code    
  118. 118. Going further  Submit bug reports  Improve the on­line documentation  Contribute transports, modules, examples  MuleForge  Blog about your Mule adventure  Let MuleSource know    
  119. 119. Books    
  120. 120. Thank you!  VIJUG  Manning Publications  VIJUG's sponsors  ... and you!    
  121. 121. Questions? david@muleinaction.com http://muleinaction.com http://dossot.net    

×