Devoxx 2011 integration-camel-cxf-servicemix-activemq

Integration with Apache ServiceMix, Camel, CXFand ActiveMQCharles Moulliard – Gert Vanthienen Apache Committer FuseSource

Speaker : Charles Moulliard • Engineer in Agronomy & Master in Zoology – 8 years of experience in IT world development 1 (J2EE, .NET), specialized in new technologies web/ Internet & integraNon, Project manager in bank, ﬁnancial, telco, insurance and transport world • SoluNon Architect at FuseSource • CommiSer on projects : Apache ServiceMix, Apache Karaf (PMC) & Apache Camel TwiSer : hSp://twiSer.com/cmoulliard LinkedIn : hSp://www.linkedin.com/in/charlesmoulliard My blog : hSp://cmoulliard.blogspot.com

Speaker : Gert Vanthienen • @Apache SoYware FoundaNon – Apache Camel, Karaf, ServiceMix – Committer / PMC member – Apache Member • @FuseSource – Member of the engineering team – Mainly working on FUSE ESB

Agenda • IntegraNon – introducNon • PresentaNon of projects -‐ Camel, CXF, P a AcNveMQ and ServiceMix r • Architectures presentaNon t I • High availability, scalability and clustering P a• PresentaNon of Fuse IDE r • Fabric new Strategy for provisioning and cloud t II

Agenda • Integra(on – introduc(on • PresentaNon of projects -‐ Camel, CXF, P a AcNveMQ and ServiceMix r • Architectures presentaNon t I • High availability, scalability and clustering P a• PresentaNon of Fuse IDE r • Fabric new Strategy for provisioning and cloud t II

IntroducNon (1) • IntegraNon projects are really hard and pain`ul • We have to integrate diﬀerent technologies/ systems/infrastructures • Lot of standards/products/tools exist (SOA,EAI, …) and formats (EDI, Fixedlength, XML, CSV, FIX, • Their implementaNons are resource and Nme consuming • SOA, BPEL and EAI have not been successfull

IntroducNon (2)  Apache foundaNon proposes alternaNve projects to simplify « integraNon project development » • Apache Camel – integraNon « Swiss knife » toolbox • Apache CXF – webservice/REST stack • Apache AcNveMQ – middleware layer • Apache ServiceMix – Container for IntegraNon project Apache ODE (BPEL), Not covered Apache Tuscany (SCA) Apache Synapse (ESB – XML)

Agenda • IntegraNon – introducNon • Presenta(on of projects -‐ Camel, CXF, P a Ac(veMQ and ServiceMix r • Architectures presentaNon t I • High availability, scalability and clustering P a• PresentaNon of Fuse IDE r • Fabric new Strategy for provisioning and cloud t II

Camel • Birthdate -‐ March 2007 • OpenSource IntegraNon Framework implemenNng EIP paSerns (book published in 2005 by Gregor Hohpe) • Why EIP  To provide word deﬁniNon, grammar and language when designing integraNon projects • Camel uses a Domain Speciﬁc Language transposing the EIP paSerns into Java, XML

Camel • Some of its features/characterisNcs – In memory bus   AlternaNve to JBI using NMR – Route objects  can be XML, File, Stream, Bytes – TransacNonal architecture – Sync/Async exchanges – Threads Management – Error and excepNon handling

Camel (1) • More than 50 paSerns are implemented hSp://camel.apache.org/enterprise-‐integraNon-‐ paSerns.html

Camel (2) • More than 100 components hSp://camel.apache.org/components.html

Camel (3) • 18 data formats to simplify data exchange between layers, applicaNons connected • Example : JAXB, Bindy, SOAP • Uses marshaler (xml  object) and unmarshaler (object  xml) hSp://camel.apache.org/data-‐format.html

Camel (4) The model  Content Based Router And its DSL language

Camel (5)

Camel (6)

Camel (7)

Camel (8) URI corresponds to component name + properNes

Camel (9) • Transpose the DSL using Java • First the conﬁgure() method must created

Camel (10) • Next we create a java class extending RouteBuilder class

Camel (11) • Language is flexible and proposes different ways to define the Endpoints and predicates

Camel (12) • Integrate easily with IntelliJ, Eclipse, NetBeans

Camel (13) • Instead of Java, you can use Spring / XML DSL

Camel (14) • IDE assistance

DEMO 1 Apache Camel

CXF • Merge of 2 projects (CelNc and Apache Xﬁre) • Simplify the creaNon and deployment of webservices • 2 approaches “java to wsdl” or “wsdl to java”. • Support the following standards : – JAX-‐WS : Web Services (XML/SOAP) – JAX-‐RS : RESTfull service (JSON) – SOAP 1.1, 1.2, WSDL 1.1 – WS-‐Security, WS-‐Addressing, WS-‐RM

CXF (1) To use it, simply add cxf maven plugin to the pom of the project <plugin> <groupId>org.apache.cxf</groupId> <artifactId>cxf-codegen-plugin</artifactId> <version>${cxf-version}</version> <executions> <execution> <id>generate-sources</id> <phase>generate-sources</phase> <configuration> <sourceRoot> ${basedir}/target/generated/src/main/java </sourceRoot> <wsdlOptions> <wsdlOption> <wsdl> ${basedir}/src/main/resources/report.wsdl </wsdl> </wsdlOption> </wsdlOptions> </configuration> <goals> <goal>wsdl2java</goal> </goals> </execution> </executions> </plugin>

CXF (2) • … and configure a spring config file <beans xmlns="http://www.springframework.org/schema/beans" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns:cxf="http://camel.apache.org/schema/cxf" xsi:schemaLocation=" http://www.springframework.org/schema/beans http://www.springframework.org/schema/beans/spring-beans.xsd http://camel.apache.org/schema/cxfhttp://camel.apache.org/schema/cxf/camel-cxf.xsd"> <import resource="classpath:META-INF/cxf/cxf.xml"/> …. <cxf:cxfEndpoint id="reportIncident" address="http://localhost:9080/incident" wsdlURL="etc/report_incident.wsdl" serviceClass="org.apache.camel.example.ReportIncidentEndpoint"> </cxf:cxfEndpoint> …</beans>

CXF (3) • Plugin will generate Java classes – JAXB2 – Interface + @Webservice @WebService(targetNamespace = "http://reportincident.devoxx.fusesource.org", name = "ReportIncidentEndpoint")! @XmlSeeAlso({ObjectFactory.class})! @SOAPBinding(parameterStyle = SOAPBinding.ParameterStyle.BARE)! public interface ReportIncidentEndpoint {! @WebResult(name = "outputReportIncident",! targetNamespace = "http://reportincident.devoxx.fusesource.org",! partName = "out")! @WebMethod(operationName = "ReportIncident",! action = "http://reportincident.devoxx.fusesource.org/ReportIncident")! public OutputReportIncident reportIncident(! @WebParam(partName = "in”,! name = "inputReportIncident”,! targetNamespace = "http://reportincident.devoxx.fusesource.org")! InputReportIncident in! );! }!

DEMO 2 Apache CXF

AcNveMQ (1) • A high performance, reliable messaging fabric, supporNng JMS, C, C++,.Net, Stomp clients. Oﬀers persistence for messages and bridge between ServiceMix instances Clients connect to the Network connectors broker using connectors control how the with simple URIs broker interacts with other brokers for discovery, replicaNon, failover, clustering and store Topics and queues created dynamically. & forward. … all based on a ﬂexible Spring-‐based File & JDBC-‐based message stores core. supported.

AcNveMQ (2) • Point to Point – ender/producer sends messages to a queue. S – eceiver/consumer listens for messages from the queue R – essages are stored unNl read (or expired) M – essages can be persisted on disk, are read only once. M <<jvm>> frodo:ActiveMQ p:Producer Foo:File c1:Consumer m m 1: Producer sends a message 2: Consumer reads to the broker the message

AcNveMQ (3) • Publish to subscribe – JMS client sends message to the topic. A – he JMS broker sends message to all subscribers that T are currently alive. • AlternaNvely, durable subscrip@ons can be used so that all subscribers will receive message even if not alive at Nme of sending – essages are consumed x Nmes (relaNon 1 to many) M <<jvm>> m frodo:ActiveMQ s1:Subscriber p:Producer Foo:Topic m m s2:Subscriber

DEMO 3 Apache Ac(veMQ

ESB – ServiceMix (1) DemysNfy the concept Enterprise Service Bus • This is not →• Or a → But an exchanging pla`orm of messages, objects moving within transport layers

ESB -‐ ServiceMix (2) • Provide connectors(= endpoints) to collect informaNon and send it on the bus as messages • Allow to separate the business layer from those that will transport the informaNon and transform (marshal) it

ServiceMix (3) • erviceMix – Project of the Apache FoundaNon S• tarted in 2005 to implement the Java Business S IntegraNon speciﬁcaNon • ecoupling of the components (business, D technical) by using XML messages • eﬁne a model to package/deploy applicaNons D (SU/SA) • ayer transport is called  Normalised Messaging L Router

ServiceMix (4) Message is ACK or ERR Routing isdeﬁned into Normalized = the SU XML exchanges

ServiceMix (5) • JBI approach is interesNng but restricNve – Everything must be XML format – EncapsulaNon of the rouNng – Components dvlpt (25 vs more than 100 for camel) – Few support from IT actors (IBM, Oracle, …) to improve exisNng spec 1.0  2.0 – Parallel with EJB story ….

ServiceMix (6) • Decision has been taken 3 years ago by members of project to – Become independent of JBI speciﬁcaNon – Use « modularity » for jars management, versioning based on OSGI, OSGI EE speciﬁcaNons – Adopt Camel as the new rouNng engine

ServiceMix (7) • erviceMix 4 S• ased on an OSGI kernel B – Apache Felix – Eclipse Equinox • ecomes an IntegraNon B Pla`orm for open-‐source – Camel – CXF – AcNveMQ, – Aries (JPA, TransacNon, JNDI, Web)

Karaf – Heart of ServiceMix (8) • irthdate - 16th of June 2010 B• untime using OSGI R• rovide a lightweight container where P  Java applications  Camel routes  WebServices can be deployed

Karaf – Heart of ServiceMix (9) <<jvm>> :ServiceMix4 Container Container Container Container Container Container Lightweight container (Apache Karaf) OSGI Kernel (Apache Felix / Eclipse Equinox) Java Boot (Register as service - Wrapper)

Karaf – Heart of ServiceMix (10) • The pla`orm is « modular » because we can select the librairies, containers that we would like to use Jetty, Pax Web Camel, JBI Integration Web CXF WebService IoC Spring, Blueprint ActiveMQ Middleware Java

Karaf – heart of ServiceMix (11) • haracterisNcs : C  AdministraNon console (local, remote, ssh, web, jmx)  Provisioning system (features)  Hot deployment and conﬁguraNon management  Instances management  Security integraNon (JAAS → ldap, jdbc, ﬁle)  Logging management (log4j, logger, commons logging, ….)

DEMO 4 Apache Karaf/ServiceMix

The big picture <<supports transport>> <<supports transport>> ActiveMQ <<implements ﬂow with>> ServiceMix CXF <<deploy into>> <<supports activemq endpoints>> <<deploy into>> <<supports cxf endpoints>> Camel <<supports jbi endpoints>> <<deploy into>>

Agenda • IntegraNon – introducNon • PresentaNon of projects -‐ Camel, CXF, P a AcNveMQ and ServiceMix r • Architectures presenta(on t I • High availability, scalability and clustering P a• PresentaNon of Fuse IDE r • Fabric new Strategy for provisioning and cloud t II

Architecture -‐ Messaging • The bus operates the exchanges between the endpoints using Camel, NMR, JBI <<jvm>> FTP Web Service ServiceMix4 BUS Bean<<MyBean>> from(«Yp://server/dir») from(«cxf:bean:myWS») .beanRef(«myBean»); .beanRef(«myBean») ;

Architecture -‐ Java • The OSGI pla`orm proposes a Service Registry (comparable to JNDI) where "interfaces" are registered • Clients access to the service using lookup mechanism <<jvm>> A ServiceMix4 « Interface » Service DAO PoJo Hibernate/JPA C B OSGI registry <bean id="incidentServiceTarget" class="org.apache.camel.service.impl.IncidentServiceImpl"> <property name="incidentDAO"> <osgi:reference interface="org.apache.camel.dao.IncidentDAO"/> </property> </bean>

Architecture -‐ Messaging + Java • Decoupling of the "services" from integraNon layer <<jvm>> ServiceMix4 « Interface » Service C DAO PoJo Hibernate/JPA OSGI registry B A from(« cxf:bean:myWS ») Bean Web Service .beanRef(« myBean », « validate ») .beanRef(«org.devoxx.jpaService») .beanRef(« myBean », « generateResponse »);

Architecture -‐ Messaging + Java + Web • Can be web pla`orm, it uses  JeSy Web ApplicaNon Server, Pax Web – a tooling for war and OSGI HTTP Service to register Servlet, … <<jvm>> ServiceMix4 « Interface » Service Hibernate/JPA DAO PoJo A Web ApplicaNon Bundle OSGI registry OSGI RunNme Bean Web Service

Architecture – OSGI limitation• CamelContext are isolated due bundles classloader• They can’t communicate directly using direct://, vm:// component <<jvm>> :ServiceMix4 FTP WebService BUS from(Yp) from(direct://order) .to(direct:// .beanRef(“order”) order) Bundle A Bundle B 56

Architecture -‐ LimitaNon • NMR is required to bridge routes <<jvm>> :ServiceMix4 FTP Bean Order Camel Context from(Yp) from(NMR://order) .to(NMR://order) .beanRef(“order”) Bundle A Bundle B NMR

Architecture -‐ Extend • When we need asymetric deployment because CPU, memory is not suﬃcient then  we use AcNveMQ to interconnect the instances and persist messages

Agenda • IntegraNon – introducNon • PresentaNon of projects -‐ Camel, CXF, P a AcNveMQ and ServiceMix r • Architectures presentaNon t I • High availability, scalability and clustering P a• PresentaNon of Fuse IDE r • Fabric new Strategy for provisioning and cloud t II

High-‐availability -‐ AcNveMQ • 2 mechanisms pure Master/Slave & Shared message store <<jvm>> <<jvm>> frodo:ActiveMQ frodo:ActiveMQ Master state The brokers compete replicated with to acquire lock on the slave the shared system (ﬁle or DB) <<jvm>> <<jvm>> samwise:ActiveMQ samwise:ActiveMQ

AcNveMQ -‐ Network of brokers • A clustered, highly available approach with 2 machines Network <<jvm>> <<jvm>> Frodo:ActiveMq Of Frodo:ActiveMq Brokers Master Slave <<jvm>> <<jvm>> Frodo:ActiveMq Frodo:ActiveMq Server 1 Server 2

High Availability -‐ ServiceMix • Only one mechanism is available – Lock based (File, DB) • Bundles can be started or not AcNve/AcNve/ AcNve Passive <<jvm>> <<jvm>> frodo:SMX4 frodo:SMX4 Master Master Slave Slave <<jvm>> <<jvm>> samwise:SMX4 samwise:SMX4

High-‐Availability – All • Combine AcNveMQ & ServiceMix + Network of brokers Camel -‐ FTP SMX4 -‐ Master BUS Headquarter queue:order from(“Yp”) Bean .to(“amq:queue:order”) queue:order queue:order NetWork of BUS Master Broker from(“amq:queue:order”) Regional oﬃces .beanRef(“order”) Bean Camel -‐ WS queue:order BUS BUS Slave queue:order from(“amq:queue:order”) .beanRef(“order”) from(“cxf”) .to(“amq:queue:order”) SMX4 – Slave

Fuse Fabric Strategy • IntegraNon – introducNon • PresentaNon of projects -‐ Camel, CXF, P a AcNveMQ and ServiceMix r • Architectures presentaNon t I • High availability, scalability and clustering END OF PART I A break is welcome !

Fuse IDE & FabricNew Strategy to build integration projects

Fuse Fabric Strategy • IntegraNon – introducNon • PresentaNon of projects -‐ Camel, CXF, P a AcNveMQ and ServiceMix r • Architectures presentaNon t I • High availability, scalability and clustering P a • Presenta(on of Fuse IDE r • Fabric new Strategy for provisioning and cloud t II

DEMO Fuse IDE

Fuse Fabric Strategy • IntegraNon – introducNon • PresentaNon of projects -‐ Camel, CXF, P a AcNveMQ and ServiceMix r • Architectures presentaNon t I • High availability, scalability and clustering P a • PresentaNon of Fuse IDE r • Fabric new Strategy for provisioning and cloud t II

Fuse Fabric Strategy• Why Fuse Fabric ? – Integration” projects are really hard to install, “ configure, run and specially on OSGI platforms – istribution of workload/process is not an D easy task – ackaging and deployment must be P simplified – e want to make it easy! W

What is Fuse Fabric?• http://fabric.fusesource.org/• Open source software for configuring, provisioning & running Fuse and Apache software on any machines – physical, virtual, private, public, private+public cloud or not etc• Keeps you DRY from those rainy clouds :)• Weave your containers into an easy to manage fabric

Fuse Fabric: What does it solve?• Fact : – Pay twice “dependencies” creation (maven P and OSGI) A – maven-felix-pluglin simplifies bundle C creation (*) K BUT this is still a complex process A G • Solution : I • Fab (Fuse Archive Bundle) N • Use pom.xml dependencies instead of G MANIFEST to deploy bundles

Fuse Fabric: What does it solve?• Fact – OSGI : Karaf – ServiceMix allows to create new instances and D administrate them EBUT P L • Instances are not O “cloned” Y • Configurations must be M updated (manually) E • No Central N Deployment platform T

Fuse Fabric: What does it solve?• Fabric will help us to D create E instances and P manage them L – Locally O – Remotely (ssh) Y – In the cloud M E N T

Fuse Fabric: What does it do?• Scalability of HTTP, WebServices, RESTfull services depend on external infrastructure S (web farm, loadbalancer, hardware, …) C• Configuration changes are required (IP A address, port number, …) to deploy new L servers/services A B • Solution : I – irtualization of services, endpoints, brokers V L – iscovery of services, endpoints and D I message brokers T Y

What do we need ?• Registry = repository of information(s) organised in a tree model• Agent = process waiting instructions about things to do on servers (karaf, servicemix, activemq)• Profile = list of tasks to do like – Insert component(s) – Setup a repository – Configure endpoint

Registry• Based on Apache ZooKeeper – Is a Centralized Coordination Service, distributed and highly reliable – Uses a directory & file based abstraction for looking at entries

Agent• Feature deployed per Karaf/ServiceMix container• Agent connects to the registry to discover “tasks, instructions”• Agent registers important informaNon into the registry • its locaNon, how to manage it via JMX etc • Agent can control child Karaf containers or child processes

Profiles• Allow for easy group configuration & control – ike using Access Control Lists for security or LDAP l trees• Contains artefacts definition• An agent is started with one or more profiles associated with it• You can then easily control a group of processes• Profiles support multiple inheritance to simplify configuration work• Profiles are versioned for easy rolling update / rollback

Profiles• Allow for easy group configuration & control – ike using Access Control Lists for security or LDAP l trees• Contains artefacts definition – Bundle – Features = collection of bundles – Repository = features xml file – Configuration = parameters of the endpoints, …• An agent is started with one or more profiles associated with it

Profiles• Registry entries for a given agent: – fabric/configs/agents/{agent} / – fabric/configs/versions/{version}/agents/{agent} / – fabric/configs/versions/{version}/profiles/{profile} /• Profiles support multiple inheritance to simplify configuration work• Profiles are versioned for easy rolling update / rollback

Camel Fabric• Expose / Virtualise a endpoint into the fabric space• Allow to invoke a remote endpoint• Loadbalance requests

DEMO Fabric, Zookeeper. Provisioning and loadbalancing (ElasNc services)

D-OSGi Fabric• Implementation of distributed OSGI defined in Enterprise OSGI Specification v4.2• Allow to register “Interfaces” and look-up them from remote instances• Use a TCP channel to send bytes over the wire between jvm of fabric karaf instances• Services can be distributed everywhere and easily  faciltating the scalability

D-OSGi Fabric (next)

DEMO Distributed OSGI services

ActiveMQ Fabric• Message brokers automatically discovery each other• Clients use connection URI “fabric:default” to connect to a message broker• If you have multiple logical ActiveMQ clusters just name them in the URI e.g. – abric:us-west, fabric:us-east, fabric:europe f• Workload can be shared between brokers (Apache Apollo)

ActiveMQ Fabric (next)

Other Fabrics• CXF Fabric – dds a feature to the CXF bus for discovery & load a balancing• ServiceMix Fabric – llow local NMR to use remote endpoints when no local a NMR endpoint available• Process Fabric – tart & monitor processes s – eep processes running across machines k

References • IntegraNon -‐ Camel : hSp://camel.apache.org • ESB -‐ ServiceMix : hSp://servicemix.apache.org • RunNme -‐ Karaf : hSp://karaf.apache.org • WebService -‐ CXF : hSp://cxf.apache.org • Broker -‐ AcNveMQ : hSp://acNvemq.apache.org • OSGI EE -‐ Aries : hSp://aries.apache.org • FuseSource : hSp://fusesource.com • EIP : hSp://www.enterpriseintegraNonpaSerns.com

References• Fabric web site : – Sp://fabric.fusesource.org/ h – Sps://github.com/fusesource/fabric h• Apache Zookeeper project – Sp://zookeeper.apache.org/ h• Demo – Sps://github.com/fusesource/fabric/tree/master/ h fabric-‐examples/fabric-‐camel-‐cluster-‐loadbalancing – Sps://github.com/fusesource/fabric/tree/master/ h fabric-‐examples/fabric-‐camel-‐dosgi

Any Questions? • twitter: @cmoulliard@gertv #fusenews • http://fusesource.com A Progress SoYware Company

http://paper.li	15
http://a0.twimg.com	6
http://tweetedtimes.com	2

Devoxx 2011 integration-camel-cxf-servicemix-activemq

by Moulliard on Nov 15, 2011

Accessibility

Categories

Tags

Upload Details

Usage Rights

3 Embeds 23

Statistics

Devoxx 2011 integration-camel-cxf-servicemix-activemq — Presentation Transcript