Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

The forgotten route: Making Apache Camel work for you

137 views

Published on

This is a classic example of older technology not being used to its fullest, which Justin proves by walking through little-known configuration and optimization tricks that get data flowing reliably and efficiently – even for today’s complexity and scale. This session covers:

A – Camel basics, understanding Exchanges, Routes, and how to implement EIPs with them
B – Examples of real implementations of common EIPs like Content Based Routers and Recipient Lists
C – Integration of Camel with common endpoints, like JMS, FTP, and HTTP

Published in: Software
  • Be the first to comment

  • Be the first to like this

The forgotten route: Making Apache Camel work for you

  1. 1. 1© 2018 Rogue Wave Software, Inc. All Rights Reserved. 1 Making Camel Work for You • Justin Reock • Sr. Director and Lead Architect • Rogue Wave OSS Support
  2. 2. 2© 2018 Rogue Wave Software, Inc. All Rights Reserved. 2 Who’s speaking to me?
  3. 3. 3© 2018 Rogue Wave Software, Inc. All Rights Reserved. 3 Justin Reock • Sr. Director and Lead Architect of our OSS Support Team • Way too much experience with ActiveMQ and Camel • Developed our formal instructor-led training • Numerous professional services and training engagements • Early adopter of JMS and Camel • Major FL/OSS evangelist!
  4. 4. 4© 2018 Rogue Wave Software, Inc. All Rights Reserved. 4 What are we going to solve?
  5. 5. 5© 2018 Rogue Wave Software, Inc. All Rights Reserved. 5 Quick exercise… • I’m going to show you three slides • They all have something in common…. • See if you can figure out what it is!
  6. 6. 6© 2018 Rogue Wave Software, Inc. All Rights Reserved. 6 The Chupacabra…
  7. 7. 7© 2018 Rogue Wave Software, Inc. All Rights Reserved. 7 A magical unicorn…
  8. 8. 8© 2018 Rogue Wave Software, Inc. All Rights Reserved. 8 BillingFulfillment CRME-Commerce A homogeneous enterprise landscape…
  9. 9. 9© 2018 Rogue Wave Software, Inc. All Rights Reserved. 9 Answer  None of these things exist in real life!
  10. 10. 10© 2018 Rogue Wave Software, Inc. All Rights Reserved. 10 A Word on ESBs • An Enterprise Service Bus is an architectural design pattern that focuses on providing federation between systems using a common runtime • Ideally, a Service Bus will provide loosely coupled endpoints to which various, heterogeneous systems can communicate with one another via their native frameworks • So, for instance, a .NET SOAP-driven application should be able to send data along the bus, which could be received by, say, a Java-based JMS destination • The ESB will provide payload and protocol normalization to facilitate that communication • Loose coupling will ensure that services can remain unaware of federated services’ languages and frameworks
  11. 11. 11© 2018 Rogue Wave Software, Inc. All Rights Reserved. 11 Examples of ESBs
  12. 12. 12© 2018 Rogue Wave Software, Inc. All Rights Reserved. 12 Common Enterprise Architecture Problems • No Universal Rules and Routing – Many application frameworks and protocols exist, but they are fragmented – They may use different protocols, data formats, and remote communication methods • Need for scale – Business tend to grow their infrastructure and federate more systems – As they do, the need for better scalability presents itself • Heterogeneous clients – Server frameworks aside, the application layer also tends to be highly fragmented – You will no doubt have applications in different languages that have a need to communicate with one another
  13. 13. 13© 2018 Rogue Wave Software, Inc. All Rights Reserved. 13 Common problems with ESBs • Custom Adapters – When not using something like SOAP, often there is a need to customize an ESB client to fit into a particular framework – This refutes an ESB’s notion of loose coupling and platform agnosticism • Non-standard implementations – Often an ESB provider will implement a protocol “their way” which can be problematic for more standard platforms • Disorganized Dependencies – With all of the various transports, frameworks, and protocols an ESB should be capable of delivering, our dependency libraries can get messy • Deployment Governance – And with so many developers contributing to an organization’s ESB logic, controlling all of that deployment creates its own set of challenges
  14. 14. 14© 2018 Rogue Wave Software, Inc. All Rights Reserved. 14 But isn’t Camel an ESB? • No! Camel can form an essential part of an ESB, like it does in the Apache Servicemix product • But it is a standalone normalized messaging framework • Camel is an example of one part of a Message Oriented Middleware system • It provides a routing and choreographing framework for moving data around an enterprise • Message Oriented Middleware often consists of several different platforms orchestrating in concert
  15. 15. 15© 2018 Rogue Wave Software, Inc. All Rights Reserved. 15 But isn’t Camel an ESB? • No! Camel can form an essential part of an ESB, like it does in the Apache Servicemix product • But it is a standalone normalized messaging framework • Camel is an example of one part of a Message Oriented Middleware system • It provides a routing and choreographing framework for moving data around an enterprise • Message Oriented Middleware often consists of several different platforms orchestrating in concert
  16. 16. 16© 2018 Rogue Wave Software, Inc. All Rights Reserved. 16 Message oriented middleware • Technically, MOM is any platform that sends and receives messages between distributed applications • This allows for asynchronous processing, and normalization of data exchanges • Clients connect to a messaging provider, and send and receive messages via that provider
  17. 17. 17© 2018 Rogue Wave Software, Inc. All Rights Reserved. 17 What’s it good for? • Data normalization makes it a lot easier for disparate applications to trade information • Just like in web services, heterogeneous systems can agree on a data “contract” and send and receive data in that format • All the application needs is a client compatible with the messaging provider • And since messages are allowed to queue up on the message provider, it also makes it very easy to perform asynchronous processing • Most message providers offer additional benefits, like guaranteed delivery, options for traffic shaping, high availability, and scaling
  18. 18. 18© 2018 Rogue Wave Software, Inc. All Rights Reserved. 18 What is Camel?
  19. 19. 19© 2018 Rogue Wave Software, Inc. All Rights Reserved. 19 The repetitive world of integration • So, data routing and transformation in the Enterprise lends itself to a lot of what you would call “boilerplate” code • Boilerplate text, in publishing, is text that can be used and reused in many places • Boilerplate code is the same concept • How many times have you written code to open up a web server, push a file to an ftp site or drop a message into a message broker? • Have you ever felt that you’re just writing different versions of the same thing over and over again? • You’re not alone • Enter the concept of an Enterprise Integration Pattern…
  20. 20. 20© 2018 Rogue Wave Software, Inc. All Rights Reserved. 20 What’s Camel? • Camel is an open source MOM language developed by Apache and based on Enterprise Integration Patterns • Book by Gregor Hohpe and Bobby Woolf • Centers around 60+ common patterns found in Enterprise integration projects • Defines these common patterns in a vendor neutral way • Camel aims to provide a language which implements these patterns, in a style modeled after the UNIX pipeline
  21. 21. 21© 2018 Rogue Wave Software, Inc. All Rights Reserved. 21 Project notes • Camel is maintained by the Apache Camel community • It is a top-level project hosted at http://camel.apache.org • The project’s chief maintainer is Claus Ibsen, author of Camel in Action • As of August of 2017, the project has had almost 400 distinct committers, spanning over 30,000 commits • It is written primarily in Java, and also contains some Scala code • The project is licensed under the ALv2 (Apache License v2.0), which is considered a permissive license • So, Camel has been folded in to many other projects, including the Mule ESB and the Switchyard project • Camel can be run standalone, packaged into a WAR, or deployed as an OSGI bundle • A POJO syntax as well as a Spring DSL XML syntax are made available for developers
  22. 22. 22© 2018 Rogue Wave Software, Inc. All Rights Reserved. 22 Integration patterns • Content Based Router – Where a message is routed to disparate endpoints based on message criteria
  23. 23. 23© 2018 Rogue Wave Software, Inc. All Rights Reserved. 23 Integration patterns • Content Enricher – In which a basic message enters the pattern, takes data from another source, and then adds it to the message
  24. 24. 24© 2018 Rogue Wave Software, Inc. All Rights Reserved. 24 Integration patterns • Recipient List – A message is broadcast to n channels based on message criteria. When this message is broadcast to all channels regardless, it becomes a Multicast pattern
  25. 25. 25© 2018 Rogue Wave Software, Inc. All Rights Reserved. 25 Integration patterns • Scatter-Gather – In which messages are broadcast out to disparate endpoints and results are aggregated into a single message
  26. 26. 26© 2018 Rogue Wave Software, Inc. All Rights Reserved. 26 How does it work? • Camel uses a repeatable, normalized concatenation of “Processor” and ”Message” objects in a group called an Exchange • There is an ”In” message, a “Processor”, and an “Out” message Processor In Out
  27. 27. 27© 2018 Rogue Wave Software, Inc. All Rights Reserved. 27 How does it work? • And, you can chain these Exchanges together – just like piping commands through UNIX – and form a Camel Route • The “Out” message of a previous Exchange becomes the “In” message of a new Exchange: P In Out P In Out P In Out
  28. 28. 28© 2018 Rogue Wave Software, Inc. All Rights Reserved. 28 Camel magic • Now here’s the magical part – a Processor can be anything you want • Components in Camel work by creating Processors that perform various business functions • The components can be chained together to build Integration Patterns • For instance, you can have an “FTP” processor that takes the In message and initiates an FTP session • Or you can have an “SMTP” processor that takes the In message and sends an email with it • This is in addition to Camel’s many built-in patterns, such as LoadBalance, Multicast, DataSets, etc. • And there are a lot of components: http://camel.apache.org/components.html
  29. 29. 29© 2018 Rogue Wave Software, Inc. All Rights Reserved. 29 Camel components Uhm, yeah, the list isn’t gonna fit… The point is there are a lot!
  30. 30. 30© 2018 Rogue Wave Software, Inc. All Rights Reserved. 30 Embedded scripting languages • Camel also contains a number of embedded script languages that can be used for various purposes • Simple: The Simple query language can be used to introspect message headers, payloads, and properties, and is useful when routing messages based on their content • Xpath: The XPath query language will let you traverse XML structures and query for attribute and entity values, which is great for routing messages based on their XML content • JSONPath: Just like XPath, except for JSON-structured messages • Groovy: The Groovy scripting language can be used for more advanced data introspection and manipulation • JavaScript: Yes, you can do entire processors in JavaScript
  31. 31. 31© 2018 Rogue Wave Software, Inc. All Rights Reserved. 31 Camel source • Camel routes can be coded in Java POJO • Or by using Camel Spring DSL XML files from("activemq:foo").filter(). xquery("//foo"). to("activemq:bar") <route xmlns="http://camel.apache.org/schema/spring" id="TimerClient"> <from uri=”activemq:topic:broadcastData"/> <setBody> <constant>This is a test JMS message.</constant> </setBody> <to uri="activemq:Test_ActiveMQ_Route.inQueue"/> </route>
  32. 32. 32© 2018 Rogue Wave Software, Inc. All Rights Reserved. 32 Camel deployment • Camel can be deployed in a number of ways: • It can be deployed as a Java Web App using Spring or POJO • Standalone Camel routes can also be deployed using Maven and Camel • Spring Boot can launch Camel routes • Camel can be embedded within another Java application • Or within a Java-based container such as Jetty, Tomcat, JBoss/Wildfly, and others • ActiveMQ contains the Camel core libraries and can launch and deploy Camel routes • And OSGI containers like Karaf or its Servicemix counterpart can launch Camel routes • Standalone Camel routes scale very nicely in microservice containers as well
  33. 33. 33© 2018 Rogue Wave Software, Inc. All Rights Reserved. 33 Camel development • Major IDEs like Eclipse and IntelliJ can ease Camel development by allowing routes to execute within the IDE • As of Camel 2.16, comprehensive in-line tools are available which allow auto-completion and even list every parameter of an available component
  34. 34. 34© 2018 Rogue Wave Software, Inc. All Rights Reserved. 34 Camel development • But HawtIO is my current favorite way to prototype with Camel • HawtIO allows for visual debugging of routes, tracing of message lifecycle, diagram driven development, as well as monitoring
  35. 35. 35© 2018 Rogue Wave Software, Inc. All Rights Reserved. 35 Camel monitoring • Camel has a robust set of JMX instrumentation beans which can be looked at to gain more information on the route • Any JMX-capable enterprise monitoring solution will be a good fit for Camel
  36. 36. 36© 2018 Rogue Wave Software, Inc. All Rights Reserved. 36 So Camel is pretty awesome… • Camel provides many benefits for enterprise messaging applications • It comes with a free library of over a hundred integration components, as well as a powerful framework for creating your own components • Camel routes can be visually debugged, similar to jBPM or BPEL • Camel cuts down on boilerplate code enormously, by providing Spring-based integration components for popular applications and frameworks • Camel is FOSS under Apache 2.0 License • As Camel improves its underlying Spring libraries, you can take advantage of better functionality with very little if any development effort • Camel’s syntax is very easy to understand and provides a low learning curve for developers
  37. 37. 37© 2018 Rogue Wave Software, Inc. All Rights Reserved. 37 Demo: Content based router • In this example, we will be using a very simple content based router • In fact, we’ll be using the Simple query language to introspect a JMS header in a message, and routing it to a destination based on the contents of that header • We will use Camel deployed directly into ActiveMQ, and we’ll write it from scratch against a clean ActiveMQ install
  38. 38. 38© 2018 Rogue Wave Software, Inc. All Rights Reserved. 38 Demo: File content translator • Here we will read the contents of new files entering into a file directory, and convert these files into JMS messages
  39. 39. 39© 2018 Rogue Wave Software, Inc. All Rights Reserved. 39 Demo: JMS to JMS Bridge • We will use Camel to connect two ActiveMQ instances to one another • Messages will be consumed from a destination on the external instance, and produced into another destination on the internal instance
  40. 40. 40© 2018 Rogue Wave Software, Inc. All Rights Reserved. 40 Claim check pattern • Here we will implement a basic claim check pattern • Messages will be stored in a local Postgres database, the payload contents will be altered in the route, and then the original payload will be retrieved
  41. 41. 41© 2018 Rogue Wave Software, Inc. All Rights Reserved. 41 Questions? Feel free to reach out after, too! justin.reock@roguewave.com

×