Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.
Using Apache Camel a bit like AKKA 
And debunking some SOA stuff 
Johan Edstrom 
SOA Executive and Apache developer. 
Apac...
Savoir Technologies - This is where we started
Where are we now?
Well, what happened there? 
We kinda started focusing exclusively on scalability 
• We went from the majority of our cust...
So what is that scalability we talk of? 
The right resources in the right place 
• Tomcat doesn’t work better with 1024 t...
This is what we are trying to avoid....
What is the AKKA statement?
So where does AKKA come in? 
It could be part of your deployment in Apache Karaf 
It could be stand-alone 
What is it t...
So what does that AKKA look like?
Or maybe a bit more complicated? 
So what is going on here? 
• ActorSystem - umbrella naming 
• System bound error handli...
Are you trying to write Camel stuff? 
Sure 
• And yes I know that Camel-AKKA exists or is it AKKA-Camel now? 
• I know th...
So…. That was really cool. But…. 
All projects aren’t greenfield 
• Can we refactor? 
We have existing staff and investm...
Let’s pretend you have Apache Karaf 
This is not far from an EE container 
• Add OpenEJB if you want and you’ll have a fu...
And we’ll add Apache ActiveMQ 
This is another presentation :) 
• Jeff did that one earlier today, I hope you did attend!
And I presume ya’ll know Apache Camel 
Camel is a Domain Specific Language (DSL) focused on 
implementing Enterprise Inte...
Apache Camel
Do I need all of that? 
Nope, many solutions will need just a few things 
• jaxb, camel, jms, soap, rest and perhaps jdbc...
I’ve heard that OSGi sucks? 
Yes, It is a bit more complex to manage a full lifecycle 
• On the other hand you get reliab...
Decoupling with OSGi 
Enforces knowledge of imports and exports 
Allows us to version 
Programming model with micro-ser...
Use case #1 
We want to accept incoming SOAP requests 
We need to inspect those HTTP headers and then use them 
to route...
Verify your use-cases!
UseCase #1 outline 
Let us use some EIP pattern symbols to outline this 
• We haven’t really made any technology statemen...
What else do we want here? 
A “Dynamic” Router! 
• We’ll craft a simple header based Dynamic router from eip patterns. 
...
The “Dynamic Router” 
Yep, it is just a Java class. 
• Imagine how easily you could integrate this with.... Anything!
Why build that type of a route? 
We are able to turn “endpoints”, “routing”, “systems” or 
whatever you want to call it i...
So let’s sum up what we have at this point 
We defined a CXF (JAX-WS) Endpoint, generated stubs and 
tied it into Camel 
...
What about that testing? 
We are going to be dealing with an OSGi environment. 
• Like an EE container it makes testing a...
Testing the router 
First of, let us just do a Camel Test, we want to verify that 
the Dynamic routing works as expected....
How we’ll be using this dynamic data 
We basically can use this to write completely dynamic steps 
that will restart on a...
Proof of concept outline 
We now have the following modules 
• customer-service : A model library 
• customer-service-ws ...
So why didn’t we just slap a camel route in 
Tomcat? 
That’s why.
Otay - so what the heck did that have to do with 
AKKA 
I’m glad you asked! 
• We have, thanks to putting it on ActiveMQ ...
Do we think we can make that “Actor” 
Lets make it very generic 
Lets make it really dynamic 
Lets use the power of OSG...
Second use-case!!! 
We are gonna take what we learned and build parts of a 
dynamic ETL system - Basically CSV files
What is a mutator in this ETL system?
And how is it working? 
Yey! We are manipulating strings! 
• OMG, Such data…. 
• Kidding aside - we now have a tool where...
Lets stick these in Camel (MutationManager)
See that Supervisor destination? 
The “brian” of the whole system. 
• Checks if this is an initial record 
• Checks if th...
Then we add an OSGi - Manager 
(ServiceFactory)
This is purely config driven in OSGi
And the Master system initiation
And showing all the moving parts
Running the system
How did it get to Mock? 
System wide “Role” for that particular incoming source file 
was established, it was added last ...
And the operations? 
Runtime loaded 
• Part of a config chain - Applied as roles in a processing chain 
• Processing chai...
End result? 
100% Event driven 
100% Asynchronous 
No camel knowledge really needed 
Competing consumer - Horizontally...
And what else is Savoir doing? 
https://github.com/savoirtech/aetos 
• Cooler than Karaf :) 
https://github.com/savoirte...
We might be releasing 
Polymorphic CRUD 
• Based on CXF 
• Fully Cassandra (or whatever) backed 
• Event driven 
• Single...
And we might be looking more at AKKA :)
Questions?
Upcoming SlideShare
Loading in …5
×

Using Apache Camel as AKKA

5,787 views

Published on

From ChicagoCoders conference 2014

  • If we are speaking about saving time and money this site ⇒ www.HelpWriting.net ⇐ is going to be the best option!! I personally used lots of times and remain highly satisfied.
       Reply 
    Are you sure you want to  Yes  No
    Your message goes here
  • Follow the link, new dating source: ❶❶❶ http://bit.ly/369VOVb ❶❶❶
       Reply 
    Are you sure you want to  Yes  No
    Your message goes here
  • Sex in your area is here: ❤❤❤ http://bit.ly/369VOVb ❤❤❤
       Reply 
    Are you sure you want to  Yes  No
    Your message goes here

Using Apache Camel as AKKA

  1. 1. Using Apache Camel a bit like AKKA And debunking some SOA stuff Johan Edstrom SOA Executive and Apache developer. Apache Camel PMC Apache ServiceMix committer Original CXF Blueprint Author Cassandra client library developer Author jedstrom@savoirtech.com joed@apache.org
  2. 2. Savoir Technologies - This is where we started
  3. 3. Where are we now?
  4. 4. Well, what happened there? We kinda started focusing exclusively on scalability • We went from the majority of our customers wanting “ESB” • Or tune our “Messaging” To • Very modular software • Infrastructure design that would last a while • Key concept structures that were re-usable • We’ve gotten quite a bit of experience here over the years
  5. 5. So what is that scalability we talk of? The right resources in the right place • Tomcat doesn’t work better with 1024 threads Messaging  Writing QueueBrowsers to see where messages go isn’t awesome State driven - Not mentioned in EIP Patterns (Yet) Conversational - Not mentioned in EIP Patterns (Yet) Effectively aggregating - Don’t use a RDBMS as a mutex • select for update 2000 times a second doesn’t scale Transactionally safe - And that doesn’t have to mean XA Correctly utilizing EE • Connectionpools aren’t for cowards
  6. 6. This is what we are trying to avoid....
  7. 7. What is the AKKA statement?
  8. 8. So where does AKKA come in? It could be part of your deployment in Apache Karaf It could be stand-alone What is it that we really think is cool? • Asynchronous systems! • Actors • Supervisors • Scaling • Immutability Is that the goal for every Java shop out there?
  9. 9. So what does that AKKA look like?
  10. 10. Or maybe a bit more complicated? So what is going on here? • ActorSystem - umbrella naming • System bound error handling • Logging • Scaling • Then we introduce some EIP Patterns like “stuff”  We RoundRobin and allow it to scale  And we finally mount it
  11. 11. Are you trying to write Camel stuff? Sure • And yes I know that Camel-AKKA exists or is it AKKA-Camel now? • I know there is a Scala DSL for Camel - fixed a few things there… • What about the Scalaz stuff?
  12. 12. So…. That was really cool. But…. All projects aren’t greenfield • Can we refactor? We have existing staff and investments • Is it easy to train We want to do this in pieces • We really want to try, test, promote Could we do this with the existing EE Infra we have?
  13. 13. Let’s pretend you have Apache Karaf This is not far from an EE container • Add OpenEJB if you want and you’ll have a full stack It contains a web-services layer Dependency injection services OSGi integration and legacy (JBI) support (Don’t use that. - Seriously…) Camel for routing, transformation and mediation Aries and Spring for EE support, JNDI, TX, JPA Web-app deployment support
  14. 14. And we’ll add Apache ActiveMQ This is another presentation :) • Jeff did that one earlier today, I hope you did attend!
  15. 15. And I presume ya’ll know Apache Camel Camel is a Domain Specific Language (DSL) focused on implementing Enterprise Integration Patterns (EIPs) • Examples: multicast, splitter, content-based router, routing slip, “dynamic routers”, aggregator EIPs are implemented by Camel “Processors” • Users can develop their own processors • A processor is a fundamental building block • Bean language and bindings exists so that not a single piece of Apache Camel Imports will be necessary when integrating your existing code Camel can connect to a wide variety of integration technologies • Examples: JMS, HTTP, FTP, SOAP, File - There are ~ 180 components • Integrations are implemented by Camel “Components” • Users can develop their own components
  16. 16. Apache Camel
  17. 17. Do I need all of that? Nope, many solutions will need just a few things • jaxb, camel, jms, soap, rest and perhaps jdbc • Cut the container down to fit your needs • We don’t need to load all of the 100+ Apache Camel components • Pick and choose! Should I run that messaging solution inside the “ESB” • Entirely up to you, let us look a little deeper at that in a sec. Can I test these solutions or am I stuck with System.out.println and a remote debugger?
  18. 18. I’ve heard that OSGi sucks? Yes, It is a bit more complex to manage a full lifecycle • On the other hand you get reliable hot-deployment and things like configuration, service management and service subscriptions out of the box. Yes, package level import export can be confusing • So make sure you use packages in a nice modular way. Yes, it takes a while to learn the tooling • Then did you wake up one day just knowing that WAR or EAR layout? Yes, you need to learn the classpath and a bit about classloading • Which’ll make you a better developer in the end of the day. • Many problems in this area actually are created by applying EE classloaders to an OSGi environment.
  19. 19. Decoupling with OSGi Enforces knowledge of imports and exports Allows us to version Programming model with micro-services allows for re-use on an Api level vs code level Promotes contracts
  20. 20. Use case #1 We want to accept incoming SOAP requests We need to inspect those HTTP headers and then use them to route We want this to be dynamic and flexible  We might want to add a more complex routing engine later We want this to be modular and possible to run across multiple systems
  21. 21. Verify your use-cases!
  22. 22. UseCase #1 outline Let us use some EIP pattern symbols to outline this • We haven’t really made any technology statements here  We are going to use Apache ActiveMQ because I said so.  It could easily have been any other JMS provider. • We know we’ll have a synchronous part  Web-services by nature are. • We’ll try and make all the other parts asynchronous • We’ll also make sure that the participants don’t need strong integration or visibility of each other, thus we’ll (hopefully) end up with some pretty re-usable modules • Another good approach is to also use sequence diagrams
  23. 23. What else do we want here? A “Dynamic” Router! • We’ll craft a simple header based Dynamic router from eip patterns.  Technically it is in between a routing slip with termination and a real control channel pattern, we utilize JMS as our control channel for this.  We use this example to play with destinations from SoapUI Remember that ActorSystem in AKKA? • This dynamic router is a big part of “pretending” that we are doing that
  24. 24. The “Dynamic Router” Yep, it is just a Java class. • Imagine how easily you could integrate this with.... Anything!
  25. 25. Why build that type of a route? We are able to turn “endpoints”, “routing”, “systems” or whatever you want to call it into simple destination names. We only need one route….. We can break all these modules apart
  26. 26. So let’s sum up what we have at this point We defined a CXF (JAX-WS) Endpoint, generated stubs and tied it into Camel • The MEP (Message Exchange Pattern) is IN/OUT since we use CXF We then dropped in a router “Our Dynamic Router” • It’ll listen to a JMS queue called recognizer • Then it’ll start inspecting headers for a location
  27. 27. What about that testing? We are going to be dealing with an OSGi environment. • Like an EE container it makes testing a tad less fun.... Full blown container testing with something like SoapUI • Can be scripted and automated in Maven • Or you could BDD / Cucumber script it Full OSGi tests with Pax-Exam • Allows you to wire up a full container Simple OSGi registry tests with PojoSr • Let’s you simulate an environment so you can test BluePrint! Functional Junit/TestNG tests with Camel-Test. • Should be combined with at least one of the above!
  28. 28. Testing the router First of, let us just do a Camel Test, we want to verify that the Dynamic routing works as expected. • As you can see, none of these routes know about each other.  And if you can’t see that, they still don't. Keep this in mind!!
  29. 29. How we’ll be using this dynamic data We basically can use this to write completely dynamic steps that will restart on a configuration change from • File changes • Code changes • JMX changes
  30. 30. Proof of concept outline We now have the following modules • customer-service : A model library • customer-service-ws : The webservices endpoint • dynamic-routing : A generic module for routing • response-builder : A simple module that returns a payload • That means that all of the needed patterns are fulfilled. • Competing consumer is a natural thing in JMS - KACHIIIIING!!!! • Eventing is a natural part of Apache Camel - KACHIIIIING!!!! • Request Reply is “Handled for us” - KACHIIIIING!!!! • Dynamic config - KACHIIIIING!!!! •
  31. 31. So why didn’t we just slap a camel route in Tomcat? That’s why.
  32. 32. Otay - so what the heck did that have to do with AKKA I’m glad you asked! • We have, thanks to putting it on ActiveMQ  Immutability  Scalability  Asynchronous behavior • We have a separation of Concerns - Kinda like the Actor  Are we really there tough? • Not yet!
  33. 33. Do we think we can make that “Actor” Lets make it very generic Lets make it really dynamic Lets use the power of OSGi to “do stuff” The basic premise now is - • JMS is our “Actor System” • Camel handles Messaging and in/out for us as well as a lot of nice to have stuff like errorhandling, graceful shutdown, startup, validation and so on. • We want a “generic” way of handling say… Objects?
  34. 34. Second use-case!!! We are gonna take what we learned and build parts of a dynamic ETL system - Basically CSV files
  35. 35. What is a mutator in this ETL system?
  36. 36. And how is it working? Yey! We are manipulating strings! • OMG, Such data…. • Kidding aside - we now have a tool where all the transformations can be described, this system contains OGNL, pattern matchers, regular expression engines and as we can see the MutationConfig contains factories for invoking these guys.
  37. 37. Lets stick these in Camel (MutationManager)
  38. 38. See that Supervisor destination? The “brian” of the whole system. • Checks if this is an initial record • Checks if there is pre-cleansing needed • Then goes onto in some sub-processors talking to rules and persistence to build a chain of MutationSets
  39. 39. Then we add an OSGi - Manager (ServiceFactory)
  40. 40. This is purely config driven in OSGi
  41. 41. And the Master system initiation
  42. 42. And showing all the moving parts
  43. 43. Running the system
  44. 44. How did it get to Mock? System wide “Role” for that particular incoming source file was established, it was added last in line. The roles are just written to a Cassandra layer, the supervisor and the MutationManager executes them.
  45. 45. And the operations? Runtime loaded • Part of a config chain - Applied as roles in a processing chain • Processing chain determined from config and built by the ServiceManager(s) • Can be hot deployed and loaded • Can be changed on running processing
  46. 46. End result? 100% Event driven 100% Asynchronous No camel knowledge really needed Competing consumer - Horizontally scalable Hot-Deployable • And most of the cool stuff I can’t even show you….. • This pattern is quite well documented in OSGi, not that often combined with Camel but you end up with all of the parts that really are interesting in something like AKKA
  47. 47. And what else is Savoir doing? https://github.com/savoirtech/aetos • Cooler than Karaf :) https://github.com/savoirtech/hecate • Hector based Pojo Mapper  Object Graph  Annotations • CQL3 based Pojo Mapper  Object Graph  Annotations  True collection handling  column adapters https://github.com/seijoed/hazelcast-discovery • HazelCast based ActiveMq discovery system
  48. 48. We might be releasing Polymorphic CRUD • Based on CXF • Fully Cassandra (or whatever) backed • Event driven • Single bean wiring with entity Class
  49. 49. And we might be looking more at AKKA :)
  50. 50. Questions?

×