20100907 fuse-community-evening-adrian-trenaman-no-logo
Upcoming SlideShare
Loading in...5
×
 

20100907 fuse-community-evening-adrian-trenaman-no-logo

on

  • 2,734 views

 

Statistics

Views

Total Views
2,734
Views on SlideShare
2,734
Embed Views
0

Actions

Likes
1
Downloads
59
Comments
0

0 Embeds 0

No embeds

Accessibility

Categories

Upload Details

Uploaded via as Adobe PDF

Usage Rights

© All Rights Reserved

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Processing…
Post Comment
Edit your comment

    20100907 fuse-community-evening-adrian-trenaman-no-logo 20100907 fuse-community-evening-adrian-trenaman-no-logo Presentation Transcript

    • Architectures, best practices and use-cases for ActiveMQ, ServiceMix, Camel & CXF The pragmatic architect speaks out FUSE Community Evening, Rotterdam, 09/2010 Adrian Trenaman Senior Principal Solutions Architect, Progress Software trenaman@progress.com http://trenaman.blogspot.com Progress Education
    • Agenda  A little bit about what the FUSE family (ActiveMQ, ServiceMix, CXF, Camel) really is: • Conceptual architecture • Standards and technologies • Deployment and scaling • Experience on the road  Some tips and best practices • A non-exhaustive list!  How you can be successful with FUSE • As architects • As developers • As project managers 2 © 2010 Progress Software Corporation. All rights reserved. FUSE Community Evening Rotterdam - September 2010
    • What is FUSE?  Frequent, certified, supported releases of Apache Software • FUSE Message Broker - Apache ActiveMQ • FUSE ESB - Apache ServiceMix • FUSE Services Framework - Apache CXF • FUSE Mediation Router - Apache Camel  Unique selling points of FUSE (in the open source world): • The right license: ASL (no GPL, no LGPL, no attribution, …) • The right company: FuseSource (backed by Progress Software Corp.) provides enterprise-level support for FUSE, backed by Apache committers. • The right financial model: subscription-based, not license based. WHAT? You mean it’s not about the technology??? 3 © 2010 Progress Software Corporation. All rights reserved. FUSE Community Evening Rotterdam - September 2010
    • ServiceMix 4 - architecture, standards & technologies 4 © 2010 Progress Software Corporation. All rights reserved. SOA Fundamentals
    • Apache ServiceMix 4 - conceptual architecture  JVM-based runtime container for <<jvm>> integration and SOA. :ServiceMix4 • EIP-style integration flows • SOAP web services Integration Web RESTful Flows Services Services • REST web services • Business processes Business Reliable Business • Reliable messaging Processes Messaging Logic • Business Logic 5 © 2010 Progress Software Corporation. All rights reserved. FUSE Community Evening Rotterdam - September 2010
    • Apache ServiceMix 4 - conceptual architecture (cont’)  ServiceMix provides a uniform <<jvm>> approach for common cross- :ServiceMix4 functional concerns • Logging Integration Web RESTful Flows Services Services • Lifecycle and deployment • Configuration Business Reliable Business • Versioning & Dependency Mgmt Processes Messaging Logic • Management • Security • Transactions Logging Lifecycle / Configuration Deployment Versioning Management Security 6 © 2010 Progress Software Corporation. All rights reserved. FUSE Community Evening Rotterdam - September 2010
    • Apache ServiceMix 4 - standards and technologies <<jvm>> WSDL, SOAP, :ServiceMix4 XML, XSD, JAX- EIP (Apache Camel) or WS, JSON, JBI (ServiceMix 3) (Apache CXF) Integration Web RESTful Flows Services Services Java, JVM-based languages BPEL (Apache ODE) Business Reliable Business Processes Messaging Logic JMS (Apache ActiveMQ) Log4J, Slf4J, commons, Java Lifecycle / Logging Deployment Configuration OSGi Config Admin logging, … Versioning Management Security SSH, JAAS, OSGi HTTPS, TLS, … JMX, web-console, ssh 7 © 2010 Progress Software Corporation. All rights reserved. FUSE Community Evening Rotterdam - September 2010
    • Modular deployment with bundles and features  You can deploy almost anything <<jvm>> into ServiceMix 4 :ServiceMix4 • War, Jar, bundle, spring, …  Prefer OSGi bundles for your smx:> features:addUrl file:my-features.xml routing / integration / business smx:> features:install f1 logic • More modular design, explicit f1 f2 versioning, classpath control. • Can share classes or objects a:bundle (OSGi services) c:bundle b:bundle • Dynamic wiring of OSGi services allows live hot deployment of patches without impacting your common production deployment. my-features.xml x:bundle  Use the ‘feature’ mechanism to group and co-deploy bundles. y:bundle 8 © 2010 Progress Software Corporation. All rights reserved. FUSE Community Evening Rotterdam - September 2010
    • Getting into production…  Deployment from Maven repository is ideal… • … but remember, admins don’t like running internet-happy developer tools like Maven in production! • Be prepared to deliver artifacts via .zip or .tar.gz - there are features-based Maven plugins to help! SYSTEST UAT PROD DEV Maven .tar.gz 9 © 2010 Progress Software Corporation. All rights reserved. FUSE Community Evening Rotterdam - September 2010
    • Scaling up, scaling out… ‘to understand ServiceMix, first you must understand ActiveMQ’ 10 © 2010 Progress Software Corporation. All rights reserved. FUSE Community Evening Rotterdam - September 2010
    • FUSE Message Broker (Apache ActiveMQ)  A high performance, reliable messaging fabric, supporting JMS, C, .Net, and other frameworks. Clients connect to the broker using Network connectors with connectors control simple URIs how the broker interacts with other brokers for discovery, replication, Topics and queues failover, clustering created and store & dynamically. forward. … all based on a flexible Spring- File & JDBC-based based core. message stores supported. 11 © 2010 Progress Software Corporation. All rights reserved. FUSE Community Evening Rotterdam - September 2010
    • ActiveMQ: networked brokers  Brokers use network connectors to share consumer information and make routing decisions using “store-and- forward” • JMS clients use failover URLs or auto-discovery to connect to a live broker. <<jvm>> <<jvm>> p:Producer frodo:ActiveMQ gandalf:ActiveMQ Foo: Foo: m m 1: Producer sends 2: Broker m 3: message a message m to ‘frodo’ decides is delivered. the broker. to route message via broker c1:Consumer ‘gandalf’. c2:Consumer 12 © 2010 Progress Software Corporation. All rights reserved. FUSE Community Evening Rotterdam - September 2010
    • Aside: networked brokers allow you to create local broker clusters and implement wide-scale cross- geography architectures. 13 © 2010 Progress Software Corporation. All rights reserved. FUSE Community Evening Rotterdam - September 2010
    • ActiveMQ replication, clustering & failover Broker replication Shared message store <<jvm>> <<jvm>> frodo:ActiveMQ frodo:ActiveMQ Master state Brokers replicated to compete for slave using file system or network database connector. lock. <<jvm>> <<jvm>> samwise:ActiveMQ samwise:ActiveMQ 14 © 2010 Progress Software Corporation. All rights reserved. FUSE Community Evening Rotterdam - September 2010
    • Networks of master-slave pairs  A clustered, highly available approach <<jvm>> <<jvm>> <<jvm>> frodo:ActiveMQ gandalf:ActiveMQ merry:ActiveMQ Master Slave <<jvm>> <<jvm>> <<jvm>> samwise:ActiveMQ saruman:ActiveMQ pip:ActiveMQ 15 © 2010 Progress Software Corporation. All rights reserved. FUSE Community Evening Rotterdam - September 2010
    • … the thing is, it’s largely the same scaling model in ServiceMix 16 © 2010 Progress Software Corporation. All rights reserved. FUSE Community Evening Rotterdam - September 2010
    • ServiceMix 4: networks of master-slave pairs  Differs from AMQ only in that slaves can be ‘partially active’ <<jvm>> <<jvm>> <<jvm>> frodo:SMX4 gandalf:SMX4 merry: SMX4 Master Slave <<jvm>> <<jvm>> <<jvm>> samwise: SMX4 saruman: SMX4 pip: SMX4 17 © 2010 Progress Software Corporation. All rights reserved. FUSE Community Evening Rotterdam - September 2010
    • ServiceMix 4: embedded brokers  Embedded brokers speed up in-VM traffic, and facilitate location transparency via a ‘messaging fabric’ <<jvm>> <<jvm>> <<jvm>> frodo:SMX4 gandalf:SMX4 merry: SMX4 ActiveMQ ActiveMQ ActiveMQ Master Slave <<jvm>> <<jvm>> <<jvm>> samwise: SMX4 saruman: SMX4 pip: SMX4 ActiveMQ ActiveMQ ActiveMQ 18 © 2010 Progress Software Corporation. All rights reserved. FUSE Community Evening Rotterdam - September 2010
    • ‘The unbearable chattiness of brokers’ or ‘too many brokers spoiled the pot’ 19 © 2010 Progress Software Corporation. All rights reserved. FUSE Community Evening Rotterdam - September 2010
    • Breaking out the broker  Broker ‘meta-traffic’ will increase as more brokers are added. • Sharing info on what consumers are listening to what destinations. <<jvm>> <<jvm>>  If you have many SMX instances, master:AMQ slave: AMQ then it might be more sensible to deploy your AMQ infrastructure separately. <<jvm>> <<jvm>> <<jvm>> • This also keeps things frodo:SMX4 gandalf:SMX4 merry: SMX4 conceptually simple. • We like simple. Master Slave <<jvm>> <<jvm>> <<jvm>> samwise: saruman: pip: SMX4 SMX4 SMX4 20 © 2010 Progress Software Corporation. All rights reserved. FUSE Community Evening Rotterdam - September 2010
    • Respecting and supporting our JBI heritage 21 © 2010 Progress Software Corporation. All rights reserved. SOA Fundamentals
    • JBI - fundamentals <<jvm>> :SMX ServiceMix acts as a container for “components”, <<component>> <<component>> communicating with :FTP :EIP each other using the XML-based Normalized Message Router. NMR <<component>> <<component>> :JMS :SAXON Use well-known components like JMS, HTTP, CXF, Bean, FTP, FILE, or write your own. 22 © 2010 Progress Software Corporation. All rights reserved. FUSE Community Evening Rotterdam - September 2010
    • JBI fundamentals (cont’) <<jvm>> :SMX Build an application by configuring and wiring <<component>> <<component>> endpoints as SUs, :FTP :EIP combining them into SAs that can be deployed atomically. :Poller :Pipeline NMR <<component>> <<component>> :JMS :SAXON Endpoints are configured using xbean (Spring) configuration; :Producer :Transformer deployment artifacts are created using maven plugins. 23 © 2010 Progress Software Corporation. All rights reserved. FUSE Community Evening Rotterdam - September 2010
    • JBI fundamentals: packaging <<jar>> Each component is <<zip>> <<zip>> specialized using a SU. In ServiceMix, the jbi.xml xbean.xml jbi.xml xbean.xml endpoint(s) are defined in an xbean.xml file. Maven plugins are used to .class resources .class resources generate jbi.xml file ftp-input-su eip-su SUs can optionally contain support classes and <<zip>> <<zip>> resources such as WSDL & XSD jbi.xml xbean.xml jbi.xml xbean.xml SUs are bundled together into an SA to be deployed atomically. .class resources .class resources jms-output-su xslt-transformer-su 24 © 2010 Progress Software Corporation. All rights reserved. FUSE Community Evening Rotterdam - September 2010
    • JBI support in ServiceMix 4  Full support for JBI 1.0 <<jvm>>  Use JBI without the packaging! :SMX • Spring + JBI components. :FilePoller  Use NMR without canonical XML payload NMR • Great for bundle-to-bundle traffic.  Use Interceptors to monitor endpoints and message exchanges.  New clustering architecture: <<jvm>> • Clustered consumers write to :SMX NMR which uses a single JMS NMR queue. • Providers listen on queue using message selectors • ActiveMQ network connectors :JMSProvider provide location transparency. 25 © 2010 Progress Software Corporation. All rights reserved. FUSE Community Evening Rotterdam - September 2010
    • Riding camels… 26 © 2010 Progress Software Corporation. All rights reserved. FUSE Community Evening Rotterdam - September 2010
    • FUSE Mediation Router (Apache Camel)  Camel provides an embedded DSL (in Java & Spring) for implementing enterprise integration patterns. • The DSL uses URIs to define endpoints which are combined by form integration flows. from("activemq:topic:Quotes”) .filter().xpath("/quote/product = ‘widget’"). to("mqseries:WidgetQuotes") .filter().xpath("/quote/product = ‘gadget’"). to("mqseries:GadgetQuotes"); 27 © 2010 Progress Software Corporation. All rights reserved. FUSE Community Evening Rotterdam - September 2010
    • Riding the camel…  Camel describes your integration flow in one place, using an intuitive language. • … a big improvement on the forest of SA’s and SU’s of ServiceMix 3.  Camel supports 50+ endpoint technologies, and 35+ EIPs  Supports transactional SEDA architectures  Deploy anywhere (standalone, JEE, Tomcat, ActiveMQ, SMX, Spring) 28 © 2010 Progress Software Corporation. All rights reserved. FUSE Community Evening Rotterdam - September 2010
    • CXF: a service by any other name would smell as sweet… 29 © 2010 Progress Software Corporation. All rights reserved. FUSE Community Evening Rotterdam - September 2010
    • FUSE Services Framework (Apache CXF)  Flexibly create & deploy code-first or WSDL-first Java web services. Choose the payload (XML, SOAP, JSON) or transport (HTTP, JMS) declaratively with no impact on code. Implement services using JAX-WS or JAX- :Consumer :MyImpl RS Invoice inv WSDL @WebService = new Invoice(); Class MyImpl implements Invoicing { invoiceSvc.process(inv); XSD public void process( Invoice inv) Separation of concerns { } } 30 © 2010 Progress Software Corporation. All rights reserved. FUSE Community Evening Rotterdam - September 2010
    • Cool things about FUSE Service Framework  Code RESTful services using JAX-RS  Generate client-side JavaScript code on the fly for browser- based WS clients: • http://localhost:9000/InvoicingService?js  Deploy anywhere: • J2EE: WebLogic, WebSphere, Tomcat, Jboss • OSGi: Servicemix 4 • JBI: ServiceMix 3 • SpringFramework • Lightweight FUSE Spring Container • Standalone: public static void main(…) { } 31 © 2010 Progress Software Corporation. All rights reserved. FUSE Community Evening Rotterdam - September 2010
    • Tips and best practices… 32 © 2010 Progress Software Corporation. All rights reserved. FUSE Community Evening Rotterdam - September 2010
    • ActiveMQ - tips and best practices  Keep it simple: follow one of these well-known deployment patterns: • Lone Broker • Master Slave with shared data-store • Cluster of Master Slave Pairs • Cross Data Centre • Cross Geography  Management • Prefer secure JMX over web console, and use a professional console like FUSE HQ or Nagios. • Don’t just leave management to the administrators / operations team.  Continually test and validate performance.  Understand and plan your storage requirements 33 © 2010 Progress Software Corporation. All rights reserved. FUSE Community Evening Rotterdam - September 2010
    • ActiveMQ - tips and best practices (cont’)  Understand consequences of sync vs async messaging  Understand consequences of persistent versus non- persistent messaging • 20,000 mps, 2,000 mps, 200 mps  Understand impact of ‘fsync’ (now default in KahaDB)  Use JMS transactions to minimize your disk syncs and increase performance!  If you’re going to have a DLQ, use it. • Who is listening for errors? What are they going to do about it? - Camel: route message to DB and send email? - Camel: read message and initiate business process via WS? - Camel: route message to custom error handling application? 34 © 2010 Progress Software Corporation. All rights reserved. FUSE Community Evening Rotterdam - September 2010
    • Camel - tips and best practices  Give your camel context an id - makes JMX easier.  Give your route an id with routeId() - make JMX management easier  Use the Spring init-method and destroy-method attributes to cleanly start and stop your routes.  Parameterize your routes - no hard-coding of URIs.  Never assume that your route will work with all components. • Each producer component may creates its own message content - do you have the right converters in place?  A route should span no more than one screen of text  Add threading judiciously, as you need it. 35 © 2010 Progress Software Corporation. All rights reserved. FUSE Community Evening Rotterdam - September 2010
    • Camel - tips and best practices (cont’)  Consider explicit techniques vs. dynamic techniques • Camel does a lot of stuff dynamically (e.g. converters) - will this confuse your development team?  If deploying into FUSE ESB, deploy regularly to iron out OSGi issues. • Don’t leave it all to the last minute. 36 © 2010 Progress Software Corporation. All rights reserved. FUSE Community Evening Rotterdam - September 2010
    • CXF - tips and best practices  Use WSDL first  Think about where your WSDL is going to come from (file system, HTTP, auto-generated)  Think about security early  Make your endpoints manageable via JMX  Get your configuration out of Spring, and into properties files via property placeholders  Be mindful of 'first invocation' lazy initialization of JAX-B contexts: first invocation is slow! 37 © 2010 Progress Software Corporation. All rights reserved. FUSE Community Evening Rotterdam - September 2010
    • FUSE ESB - tips and best practices  Understand cost/benefit of deploying ActiveMQ in ServiceMix.  Deploy only what you need.  Use the OSGi Config Admin service - seamless way to get configuration via property placeholders, and allows dynamic updates of configuration  Consider what 'master-slave' means for routes, web services, and RESTful services  Consider OSGi Blueprint Services instead of Spring 3.0  Use features for manage deployment of your bundles  Ensure that features are packaged for 'maven-less' deployment in production 38 © 2010 Progress Software Corporation. All rights reserved. FUSE Community Evening Rotterdam - September 2010
    • Summing up… 39 © 2010 Progress Software Corporation. All rights reserved. FUSE Community Evening Rotterdam - September 2010
    • Adopting FUSE  Java, Maven, Spring - if you don’t have these skills, be prepared to get them.  Invest in technical steering - don’t go in the wrong direction  Nominate FUSE expert(s) on your team - then let them spread the love.  Know yourself - If you not source-friendly, then get professional help.  In the source, there is truth - don’t be afraid of the truth. 40 © 2010 Progress Software Corporation. All rights reserved. FUSE Community Evening Rotterdam - September 2010
    • Parting words from a battle-hardened consultant…  Don’t confuse enterprise integration with workflow or BPM • Use the right tool for the job at hand.  Respect the {n|cr}appy path above the happy path.  Validate performance early, and keep validating it.  Prefer Camel; use JBI as an integration point.  Never forget your customer. Give them visibility and control. • Thing about management early. 41 © 2010 Progress Software Corporation. All rights reserved. FUSE Community Evening Rotterdam - September 2010