A Workhorse Named Mule

A workhorse named Mule

The speaker  Mule in Action, coauthor  Project despot of Mule JCR Transport  Common Retry Policies  HTML Dashboard  Mule Community Committee Member  Lead of NxBRE  Judge for the Jolt Awards  And a few other stuff...

Nonagenda  eMule  Mule pre 2.2.0, EE  Tutorial  Works on my machine  Alternative configurations

Agenda  Introduction  Exploration  Construction  In Action

Introduction

What is Mule? ”Mule is a lightweight messaging framework and highly distributable object broker”

What? No ESB? No SOA?  ESB is a deployment topology  Mule can be deployed as ESB  But also: ESN, Hub and spoke...  SOA can leverage an ESB  Just an extra tool in a tool box  Mule has no dogma nor methodology  More on this later...

And what about Java?  Written in Java  Leverages zillions of Java open source libraries  Can do a lot without Java though  XML and Groovy can go a long way

Genesis  Founded by Ross Mason  SourceForge project created in April 2003  Version 1.0 in April 2005  Moved to CodeHaus  Architecture based on:  EIP  SEDA

EIP  Enterprise Integration Patterns  Hohpe & Woolf (2004)  Gave Mule its lingo  Endpoints, Transformers...  Routers, Bridge...  Gives us patterns & iconography http://www.enterpriseintegrationpatterns.com

SEDA  Staged EventDriven Architecture  Welsh (2001)  Disciplined approach to overload management  Eventdriven application  Stages connected by queues  Graceful degradation under load  By default Mule is asynchronous and SEDA

What's in a name? ”After working on a couple of bespoke ESB systems, I found that there was a lot of infrastructure work to be done before you can really start thinking about implementing any logic. I regard this infrastructure work as 'donkey work' as it needs doing for every project. I preferred Mule over Donkey and Ass just didn't seem right. A Mule is also commonly referred to as a carrier of load, moving it form one place to another. The load we specialise in moving is your enterprise information.” Ross Mason

Donkey work Integration App 1 Broker App 2 App 3

Mule work Adaptation, Transformation & Routing App 1 Mule App 2 App 3

Mule's dual nature

Mule's dual nature ”Mule is a lightweight messaging framework and highly distributable object broker”

Lightweight messaging framework Mule Your Application

Highly distributable object broker Mule Mule

Dual competition  Integration frameworks  Camel ✔ Strictly EIP  Spring Integration ✔ DI oriented  ESBs  Apache, WSO2, ✔ Web service OW, JBoss oriented  Big vendors ✔ Tooling ✔ Doodleware

Dual offer  Community Edition  Stable “enough”  Upgrades can be rocky  Enterprise Edition  Supported, stabler  Enterprise features  MQSeries connectors, retry policies  Monitoring tool (MuleHQ)

Key benefits I  Proven product  Used in many worldclass applications  Sound architecture  Drastically refined in Mule 2  Numerous protocols and extensions  From MuleSource and MuleForge  Spring & Maven oriented

Key benefits II  Good tooling  Mule IDE  Galaxy  Active community  Support, MuleForge  Shared code base CE / EE  Professional support available

Main drawbacks  Unified messaging abstraction  Acts as a potential keyhole on the real deal  Brings its own idiosyncrasies  Like any framework...  Standards based but not standards compliant  But what standards? JBI? SCA?  Not fully EIP but uses much of its lingo

Exploration

”A messaging framework”  Mule is about abstraction and adaptation  Message  Protocol  As a framework  You can code to it  Wisely  You can extend almost every aspect of it  Wisely

Message abstraction MuleMessage Payload Properties Attachment Exception

Message abstraction  Payload  HTTP body, File content  Properties  HTTP Headers, JMS properties  Attachments  SOAP parts, Email parts  Exception  Related to the current message processing

Protocol abstraction  Handled by ”transports”  Connectors and endpoints  Message adapters and transformers  Inbound interactions  Receive (event)  Request (polling)  Outbound interactions  Send (request/response)  Dispatch (asynchronous)

Protocol adaptation  At connector level  Handles interactions  At message level  Message adaptation  Raw data access  Specific additional transformation  e.g. Object ↔ JMS Message transformers

Protocol adaptation Inbound Protocol Adaptation Mule Transformation Mule Specific Message Message

Protocol adaptation Outbound Mule Transformation Mule Adaptation Protocol Message Message Specific

Unified payloads?  File transport  Stream  java.io.File  JMS transport  Map, String...  IMAP transport  String

xmlns:context="http://www.springframework.org/schema/context" xmlns:file="http://www.mulesource.org/schema/mule/file/2.2" xmlns:jms="http://www.mulesource.org/schema/mule/jms/2.2" xmlns:xm="http://www.mulesource.org/schema/mule/xml/2.2" xsi:schemaLocation=" http://www.springframework.org/schema/beans http://www.springframework.org/schema/beans/spring-beans-2.5.xsd http://www.springframework.org/schema/context http://www.springframework.org/schema/context/ spring-context-2.5.xsd http://www.mulesource.org/schema/mule/core/2.2 http://www.mulesource.org/schema/mule/core/2.2/mule.xsd http://www.mulesource.org/schema/mule/file/2.2 http://www.mulesource.org/schema/mule/file/2.2/mule-file.xsd http://www.mulesource.org/schema/mule/jms/2.2 http://www.mulesource.org/schema/mule/jms/2.2/ mule-jms.xsd http://www.mulesource.org/schema/mule/xml/2.2 http://www.mulesource.org/schema/mule/xml/2.2/ mule-xml.xsd" > <notifications dynamic="true"> Need help! <notification event="ENDPOINT-MESSAGE"/> </notifications> <global-property name="jms.url" value="vm://localhost"/> <jms:activemq-connector name="jmsConnector" specification="1.1" brokerURL="${jms.url}"/> <file:connector name="FileConnector" streaming="false" autoDelete="true" pollingFrequency="1000"> <file:expression-filename-parser/> </file:connector> <model name="jmsInboundModel"> <service name="jmsInboundService"> <inbound> <jms:inbound-endpoint topic="backup-reports" > <transformers> <jms:jmsmessage-to-object-transformer/> </transformers> </jms:inbound-endpoint> </inbound>

Mule's anatomy

Mule's anatomy Service Service Mule Service Service

Mule service  Wider definition than strict SOA  Both inbound and outbound are considered  Service can have business logic or not  Message routing is done by services  Role defined by messaging style  Four main styles

Request Response Request Client Service Response

Asynchronous Dispatch Client Service

Synchronous Request Request Client Service Service Response Response

Async Request Response Request Dispatch Client Service Service Response Response

Combined styles Request Dispatch Client Service Service Response

Other inbound styles Listen Source Service Poll Source Service Request Source Service

Mule's service anatomy Service ?

Mule's service anatomy Inbound Outbound Routing Component Routing Response

Mule's service routing anatomy Inbound Outbound Transformers Transformers Endpoints Endpoints Router Router Component Response

Endpoints  Communication ”touch points”  Configured  Locally in services  Inbound and outbound  Globally  Named  “Unbound”  Used locally in or out

Endpoints  Inbound <jms:inbound-endpoint topic="events" />  Outbound <file:outbound-endpoint path="./events" />  Global <jms:endpoint name="events.in" topic="events" /> ... <inbound-endpoint ref="events.in" />

Endpoints URI  Two equivalent endpoints: <jms:inbound-endpoint topic="events" /> <inbound-endpoint address="jms://topic/events" />  Prefer rich XML schema style  Generic style exists for extensibility  In code, URI style prevails

Routers  Routing behavior for endpoints  Smart “switching” and filtering  Aggregation, multicasting...  Configured locally in services  Different families  Inbound  Outbound  Asyncreply

Routers <inbound> <jms:inbound-endpoint topic="events" /> <jdbc:inbound-endpoint queryKey="eventsQuery"/> </inbound> <outbound> <multicasting-router> <jms:outbound-endpoint topic="events"/> <file:outbound-endpoint path="./events" /> </multicasting-router> <outbound>

Transformers  Message payload  Generic  Bytes to string, compression, XSLT...  Transport  JMS to object, String to email...  Message properties  CRUD  Round trip alter egos

Transformers  Configured  Locally on endpoints  In / response and out  Globally  Named  Used locally in or response  Sometimes invisible (transport)  Can be composed  Rolling your own is rare

Transformers <xm:xslt-transformer name="DocbookToFO" xsl-file="docbook-fo.xsl" /> ... <inbound-endpoint ref="docs.in.sync" > <transformers> <gzip-uncompress-transformer /> <transformer ref="DocbookToFO" /> </transformers> <response-transformers> <gzip-compress-transformer /> </response-transformers> </inbound-endpoint>

Components  Messaging behavior  Bridge, Log  Reflection message builder  Remote service proxy  Business behavior  Instantiation by Mule or Spring  Pure POJO or Mule aware  Can be pooled

Components <log-component /> <test:component/> <component> <singleton-object class="com.acme.Component" /> </component> <component> <spring-object bean="yourBean" /> </component> <pooled-component class="com.acme.Component" />

http://www.springframework.org/schema/beans/spring-beans-2.5.xsd http://www.springframework.org/schema/context http://www.springframework.org/schema/context/ spring-context-2.5.xsd http://www.mulesource.org/schema/mule/core/2.2 http://www.mulesource.org/schema/mule/core/2.2/mule.xsd http://www.mulesource.org/schema/mule/file/2.2 http://www.mulesource.org/schema/mule/file/2.2/mule-file.xsd http://www.mulesource.org/schema/mule/jms/2.2 http://www.mulesource.org/schema/mule/jms/2.2/ mule-jms.xsd http://www.mulesource.org/schema/mule/xml/2.2 http://www.mulesource.org/schema/mule/xml/2.2/ mule-xml.xsd" > <notifications dynamic="true"> <notification event="ENDPOINT-MESSAGE"/> </notifications> Pwned! <global-property name="jms.url" value="vm://localhost"/> <jms:activemq-connector name="jmsConnector" specification="1.1" brokerURL="${jms.url}"/> <file:connector name="FileConnector" streaming="false" autoDelete="true" pollingFrequency="1000"> <file:expression-filename-parser/> </file:connector> <model name="jmsInboundModel"> <service name="jmsInboundService"> <inbound> <jms:inbound-endpoint topic="backup-reports" > <transformers> <jms:jmsmessage-to-object-transformer/> </transformers> </jms:inbound-endpoint> </inbound> <outbound> <pass-through-router> <file:outbound-endpoint path="./data/reports" outputPattern="BACKUP-REPORT-#[function:dateStamp]"/> </pass-through-router>

Service configuration <service name="eventsWriter"> <inbound> <jms:inbound-endpoint topic="events" /> </inbound> <outbound> <pass-through-router> <file:outbound-endpoint path="./events" /> </pass-through-router> </outbound> </service>

Service configuration <service name="eventsWriter"> <inbound> <jms:inbound-endpoint topic="events" /> </inbound> <bridge-component /> <outbound> <pass-through-router> <file:outbound-endpoint path="./events" /> </pass-through-router> </outbound> </service>

Anything else?  Connectors  Transport specific configuration  Model  Shared configuration & behavior for services  Interceptors  To add crosscutting concerns around components

The mysterious Mule client

The mysterious Mule client  Why a client?  Inmemory access to an instance  Message generation  Functional testing  Remote access to an instance  Proprietary remote invoker  Raw transport usage

Mule client interactions (some)  Dispatch muleClient.dispatch("jms://queue", map, null);  Send MuleMessage response = muleClient.send("vm://channel", payload, props);  Request MuleMessage requested = muleClient.request("file://events/01.dat", 0);

Construction

What do we build?  Standalone  Embedded  Project structure  Java application  NetBoot  Web application  Repository artifacts  EJB application  JCA resource

Just ship it!  A Mule project is just a bunch of XML files  Tinker with them a little  Ship them to production  And you are good to go!  Right?

No escape from SD laws!  Practices  Middleware development  TDD  Repeatable builds  Repeatable deployments  Tools  SCM  Continuous integration

Middleware development practices  Do not get coupled with the middleware  Contain Mule related code to Mule aware classes  Transformers, interceptors  Caching  Timeouts  Efficient logging and manageability

Unit Testing  For components, transformers...  Outside of Mule  xUnit

Functional Testing  For routing, configuration...  Testscoped instance  Mule client  Test component  Substitution endpoints  Inmemory infrastructure  Jitr

Integration Testing  For Mule instance in situation  Complex because heterogeneous  Test messages  Test services  Secondary endpoints  soapUI, HermesJMS  Test Driven EAI  Hohpe & Istvanik (2002) http://bit.ly/tdeai

Other Testing  Load tests  Use JMeter  Profiling  Use the YourKit Profiler Pack

Asynchronous testing challenge

Asynchronous testing (lucky) Test Thread Mule Thread Start Mule Send test message Process message Perform assertions Stop Mule

Asynchronous testing Test Thread Mule Thread Start Mule Send test message Perform assertions Process message Stop Mule

Asynchronous testing Fixed Test Thread Mule Thread Start Mule Send test message Process message Await latch (n) Latch down Perform assertions Stop Mule http://bit.ly/tasync

Unleashing Mule

Unleashing Mule I  Generic knowledge  Maven  Spring

Maven  Mule is available as a set of artifacts  Core, transports, modules  With petazillions of dependencies  Distribution packs them all  Specific archetypes  New project  New transport  Galaxy publication

Spring  Code is independent from Spring  Configuration is fully Spring XML  With specific schemas  Use Spring configuration practices  DI, properties, multiple files...  Use Spring goodness  Specific elements, MBean exporter...  Mule's context can use a Spring parent  Bean reuse

Unleashing Mule II  Specific knowledge  VM Transport  Transport configuration  Expressions  Scripting  Entry point resolvers  Notifications

VM Transport  Inmemory named pipe  Direct or queuing  Can be disk persistent  What you can not solve with one service  You solve it with two services  And a VM Queue in between!

Transport configuration  Know the underlying transport  Understand what Mule hides & controls  Learn the configuration options  Including alternative adapters, transformers  Leverage the transportspecific properties  Inbound and outbound  Read the source code

Expressions  Dynamic properties in configuration  #[evaluator:expression]  Filters, transformers, endpoint URIs  Usually evaluated on the current message  Standalone and Mulewide evaluators exist  Many evaluators available  Groovy, XPath, Map Payload, OGNL...  Can also be invoked from code

<message-properties-transformer name="SetTo"> <add-message-property key="toAddresses" value="#[xpath:/Receipts/SendToEmail]" /> </message-properties-transformer> ... <smtp:outbound-endpoint host="${smtp.host}" port="${smtp.port}" from="${smtp.fromAddress}" subject="${smtp.subject}" transformer-refs="SetTo StringToEmailMessage" />

Scripting  Any JSR223 engine  Groovy bundled with the scripting module  Rhino bundled with JDK 1.6  Transformers, components, expressions  Good for nongeneric ones  Concise syntax (i.e. nonJava)  Inlined or standalone scripts  Spring Scripted Beans are also an option

<scripting:transformer name="toUpperCase"> <scripting:script engine="groovy"> return payload.toUpperCase() </scripting:script> </scripting:transformer> <scripting:component> <scripting:script engine="js"> if (payload.search("STATUS: CRITICAL") != -1) { message.setProperty("PRIORITY", 'HIGH') } return message </scripting:script> </scripting:component> <expression-filter evaluator="groovy" expression="message.payloadAsBytes.length>1024" />

Entry point resolvers  Secret behind Mulefree components  Locate the method to call  Several resolvers available  Reflection, array, no argument, method name  By default, Mule uses a set of resolvers  Add a transformation in front to better resolve  By type or directly by name

Notifications  Internal notification framework  Extreme diversity of events  Mule lifecycle, routing, errors, timeouts...  Standard consumers  JMX, Log4J, publish to endpoint  Custom consumers  Components, interceptors...

In Action

The tough questions

The tough questions  SOAP or REST?  Monitoring  Transactions  Auditing  Exception handling  High availability  Security  Hot reload  Scheduling  Governance  Orchestration  Performance

SOAP or REST?  SOAP transports  REST transports  CXF  Abdera  Axis  Jersey  Restlet

Transactions  Supported by a handful of transports  VM, JDBC, JMS  Mostly useful for bridging  Like Queue to Topic  Consider eventual consistency  Conversation based  Idempotent services  Compensation

Exception handling  Exceptions are handled at several levels  Connector and service  Retry policies  Different strategies  Logging, dispatch to endpoint  Transaction rollback  Exception based outbound router  Localized error handling via fallback

Security  Secure endpoints  Authentication and authorization  Secure payload  Encryption  Security manager  Can delegate to Spring Security  Can use JAAS  Transport security handled at connector level

Scheduling  Based on Quartz  In inbound router  To regularly send a fix payload  To request an endpoint (polling)  In outbound router  To schedule a delayed dispatch

Orchestration  Trivial, short lived  Message flow is the orchestration  Component binding, chaining router  Complex, long lived  Messages trigger worflow steps  jBPM (native)  Orchestrator triggers services  ActiveBPEL (via WS)

Chaining router <service name="WeatherDispatchService"> <inbound> <inbound-endpoint ref="WeatherRequest" /> </inbound> <outbound> <chaining-router> <outbound-endpoint ref="WeatherChannel" /> <jms:outbound-endpoint queue="weather" /> </chaining-router> </outbound> </service>

Monitoring  Standard Java middleware  Monitor JVM with SNMP or JMX  Monitor Mule with JMX  Monitor external channels  Queues filling up, DB not refreshed...  EE relies on MuleHQ  Based on Hyperic

Auditing  Hard with distributed heterogeneous systems  Watermark messages with correlation ID  Standard with JMS  Ad hoc with other transports  Always log correlation ID with all messages  Mule takes great care of correlation ID

High availability  Easily achieved with stateless instances  Redundancy & network load balancer  Clustering needed with stateful instances  EE will offer GigaSpaces  Terracotta  DB, in some cases  ID store for idempotent receiver

Mythical hot reload  Everybody wants it  Really?  Mule configuration can be hot reloaded  Since 2.2.2  Mule libraries are OSGi bundles  But Mule is not (yet) running on OSGi

Governance  Publicly exposed services are commitments  Required by compliance rules  Galaxy  Repository and registry  Versioning  Backwards compatibility check  Galaxy EE is HA

Governance EE

Performance

Performance I  Beware benchmarks  SEDA is tricky to tune  Many thread pools  Some not directly handled by Mule  Threading  Configured with profiles  Affected by transactions, synchrony  Follow MuleSource's methodology

Performance II  Consider streaming  Smooks for efficient processing  Tune the transports  HTTP chunking, JMS acking...  Profile first!  YourKit Profiler Pack

The big picture

To ESB or not? I  ESB in the middle of all service calls  Looks good on slides  But a big mess inside  Remember Jim Webber  “Guerilla SOA”  ESB == “Egregious Spaghetti Boxes” http://bit.ly/gsoa

To ESB or not? II  “A good ESB is a dead ESB”?  Decision points: http://bit.ly/toesbornot  “MuleSource is Open for Integration.”  Don't get confused on branding/naming  ESB is a topology not a tool  There are others...

Topology  ESB  JMS backbone  ESN  Direct service to service  Hub & Spoke  Centralized broker  Client / Server  ...

Methodology  None officially preached by MuleSource  But lots of experience with many clients  Fits with the one inhouse  Nespresso Open Architecture (NesOA)  Read MuleSource's articles  To get the gist of their approach  Pragmatism first!

Bottomup SOA ” I’ve seen firsthand that the more successful enterprises that have adopted SOA have done so at the grassroots level. (...) Certainly many of our customers choose Mule not on a strategic initiative, but because they have pockets of successful implementations and decide to work with us to help pull the pieces together.” http://bit.ly/bupsoa Ross Mason

Time to hit it!

Getting started  www.mulesource.org  Download CE, browser user guide  www.mulesource.com  Professional support  http://code.google.com/p/muleinaction/  Mule in Action source code

Going further  Submit bug reports  Improve the online documentation  Contribute transports, modules, examples  MuleForge  Blog about your Mule adventure  Let MuleSource know

Thank you!  VIJUG  Manning Publications  VIJUG's sponsors  ... and you!

Questions? david@muleinaction.com http://muleinaction.com http://dossot.net

http://coderwall.com	41
http://www.slideshare.net	18
http://www.linkedin.com	8
http://bbtest.mclennan.edu	1
http://webcache.googleusercontent.com	1
http://www.techgig.com	1
http://115.112.206.131	1

A Workhorse Named Mule

by David Dossot

Accessibility

Categories

Upload Details

Usage Rights

7 Embeds 71

Statistics

A Workhorse Named Mule Presentation Transcript