Integrating Microservices with Apache Camel
•17 likes•10,702 views
Download to read offline
Report
Slides from my Integrating microservices with Apache Camel and Fuse
Recommended
More Related Content
What's hot
What's hot(20)
Viewers also liked
Viewers also liked(6)
Similar to Integrating Microservices with Apache Camel
Similar to Integrating Microservices with Apache Camel(20)
More from Christian Posta
More from Christian Posta(20)
Recently uploaded
Recently uploaded(20)
Integrating Microservices with Apache Camel
- 1. Integrating Microservices with Apache Camel and JBoss Fuse
- 2. 2 Who? Christian Posta Principal Middleware Specialist/Architect Blog: http://christianposta.com/blog Twitter: @christianposta Email: christian@redhat.com • Committer on Apache Camel, ActiveMQ, Fabric8, PMC on ActiveMQ • Author: Essential Camel Components DZone Refcard • Frequent blogger and speaker about open-source technology!
- 3. Why do we care?
- 7. Where are we now?
- 8. • Domain modeling • Object Oriented design • Functional decomposiiton • Language specific modularity, decomposition, fault-tolerance (eg, Erlang) • Shared libraries? Decomposition techniques
- 10. • Agile methodology • Doman Driven Design • REST • Hexagonal Architectures • Pipes and Filters • Actor Model • SEDA Microservices emerged as a result…
- 11. A new term! Yay! A concept that helps describe distributed systems that organically evolve into scalable, loosely coupled, independently managed sets of services that work together to deliver business value with acceptable tradeoffs. So what are microservices?
- 12. Don’t get too caught up in the buzzword bingo; Microservices is a good concept, but it’s not itself a panacea. So what are Microservices?
- 13. • Flexible technology choices • “Smart endpoints” “dumb pipes” • Independently scalable • Decentralized, choreographed interactions • Testable • Automation, DevOps philosophy • Design for failure • Evolving design Microservice characteristics
- 14. • No silver bullet; distributed systems are *hard* • Dependency hell, custom shared libraries • Fragmented and inconsistent management • Team communication challenges • Health checking, monitoring, liveness • Over architecting, performance concerns, things spiraling out of control fast Challenges with Microservices!
- 16. Real developers ride Camels!
- 17. 1 Apache Camel Apache Camel is an open-source, light-weight, integration library. Use Camel to integrate disparate systems that speak different protocols and data formats
- 18. Apache Camel Enterprise Integration Patterns http://camel.apache.org/eip
- 19. Features ● Enterprise Integration Patterns (EIPs) ● Domain Specific Language to write “flows” or “routes” ● Large collection of adapters/components for legacy systems, B2B, and SaaS ● Strong Unit test/Integration test framework ● Expression languages ● Data Formats ● Tooling with JBoss Developer Studio
- 21. Java DSL public class OrderProcessorRouteBuilder extends RouteBuilder { @Override public void configure() throws Exception { from(“activemq:orders”) .choice() .when(header(“customer-rating”).isEqualTo(“gold”)) .to(“ibmmq:topic:specialCustomer”) .otherwise() .to(“ftp://user@host/orders/regularCustomers”) .end() .log(“received new order ${body.orderId}”) .to(“ibatis:storeOrder?statementType=Insert”); } }
- 22. Spring XML DSL <route id=“processOrders”> <from uri=“activemq:orders”/> <choice> <when> <simple>${header.customer-rating} == ‘gold’</simple> <to uri=“wmq:topic:specialCustomer”> </when> <otherwise> <to uri=“ftp://user@host/orders/regularCustomers” /> </otherwise> </choice> <log message=“received new order ${body.orderId}”/> <to uri=“ibatis:storeOrder?statementType=Insert”/> </route>
- 23. Camel - JBoss Developer Studio
- 24. • Dynamic routing options • REST DSL • Backpressure mechanisms • Loadbalancing algorithms / Circuit Breaker pattern Heavy Lifting: Camel for Microservices
- 25. • “Smart endpoints, dumb pipes” • Endpoint does one thing well • Metadata used for further routing • Really “dynamic” with rules engine (eg, Drools/BRMS) Dynamic Routing
- 26. • Content Based Router • Dynamic Router • Routing Slip • Recipient List Dynamic Routing
- 27. • Expressive way to define REST endpoints • POST, REST, PUT • Auto binding to POJOs • Plugs into Swagger for interface definition/documentation • Uses configurable HTTP engine • camel-netty-http • camel-jetty • camel-reslet • camel-sevlet (deploy into other containers) • camel-spark-rest REST DSL (2.14)
- 28. REST DSL public class OrderProcessorRouteBuilder extends RouteBuilder { @Override public void configure() throws Exception { rest().post(“/order/socks”) .description(“New Order for pair of socks”) .consumes(“application/json”) .route() .to(“activemq:topic:newOrder”) .log(“received new order ${body.orderId}”) .to(“ibatis:storeOrder?statementType=Insert”); } }
- 29. • Backpressure is a way for a service to flow control callers • Detecting when can be difficult • Need to bound processing queues in a SEDA • Take advantage of built in TCP flow control for socket/http requests Backpressure with Camel
- 30. • Throttle EIP • http://camel.apache.org/throttler.html • Blocking SEDA Queue • from(“seda:name?size=100&blockWhenFull=true) • Configure jetty/netty to use blocking acceptor queues • https://wiki.eclipse.org/Jetty/Howto/High_Load • Using Exception handling/retry and DLQ logic when getting flow controlled • http://camel.apache.org/error-handling-in-camel.html Backpressure with Camel
- 31. • Useful to keep from overloading a system (use in conjunction with backpressure if you can) • Smart loadbalancing • Sticky • Random • Failover • Circuit breaker Loadbalance/Circuit breaker
- 32. Circuit breaker Image from http://martinfowler.com/bliki/CircuitBreaker.html
- 33. Circuit breaker public class OrderProcessorRouteBuilder extends RouteBuilder { @Override public void configure() throws Exception { from(“direct:someinterface”) .loadbalance() .circuitBreaker(3, 20000L, MyException.class) .to(“ibatis:storeOrder?statementType=Insert”); } }
- 34. Apache Camel More Information ● Camel in Action ● Apache Camel Developer’s Cookbook ● Community website ○ http://camel.apache.org/
- 35. Why JBoss Fuse?
- 37. RED HAT JBOSS FUSE Development and tooling Develop, test, debug, refine, deploy JBoss Developer Studio Web services framework Web services standards, SOAP, XML/HTTP, RESTful HTTP Integration framework Transformation, mediation, enterprise integration patterns Management and monitoring System and web services metrics, automated discovery, container status, automatic updates JBoss Operations Network + JBoss Fabric Management Console (hawtio) Apache CXF Apache Camel Reliable Messaging JMS/STOMP/NMS/MQTT, publishing-subscribe/point-2-point, store and forward Apache ActiveMQ Container Life cycle management, resource management, dynamic deployment, security and provisioning Apache Karaf + Fuse Fabric RED HAT ENTERPRISE LINUX Windows, UNIX, and other Linux
- 41. • Simplifies deployments • Provides centralized configuration • Provides cluster capabilities, coordination • Service discovery • Smart load balancing • Failover • Versioning • Visualize your middleware with HawtIO http://fabric8.io
Editor's Notes
- We are going to talk about microservices, and all the goodness that buzzword entails… because you probably wouldn’t have joined this webinar with a few buzzwords… because a talk named something like “oh the same old service oriented architecture with camel” wasn’t as nifty of a title…
- Everyone wants tightly integrated business systems However, a business’s model is usually far too complex to understand as a single unit Gotta break things down and modularize and reason about things in smaller units Eric Evans: The goal of the most ambitious enterprise system is a tightly integrated system spanning the entire business. Yet the entire business model for almost any such organization is too large and complex to manage or even understand as a single unit. The system must be broken into smaller parts, in both concept and implementation. The challenge is to accomplish this modularity without losing the benefits of integration, allowing different parts of the system to interoperate to support the coordination of various business operations. A monolithic, all-encompassing domain model will be unwieldy and loaded with subtle duplications and contradictions. A set of small, distinct subsystems glued together with ad hoc interfaces will lack the power to solve enterprise-wide problems and allows consistency problems to arise at every integration point. The pitfalls of both extremes can be avoided with a systematic, evolving design strategy.
- If you’re running everything on a single machine that is infinitely scalable… then most of this talk isn’t really for you… Building distributed systems is HARD And the single most difficult thing to get right is architecture… regardless of whether your doing monolith deployments or micro deployments, etc Make lives easier! Be able to maintain an application, or set of applications, and evolve it…. Don’t want to have to pay technical debt… Any time you make decisions, you offset and trade technical debt… it’s baseline technical debt.. Can’t do anything about that.. Any bad descisions you make to start, you start paying debt and it becomes exponential… Debt is always going to grow.. Maintainability ------------------ Swapping out pieces for off the shelf? Or the other way around, swapping out for cheaper, commodity pieces Scaling Bug fixes, dependency ripple affect But we are developing soft systems, not rigid buildings or structures.. Our requirements change frequently, and the way the applications we build are used in diferent ways Not only does the business landscape constantly change and force changes, but even without those constraints, once our software is built, we find that people use it or need it for something different than they thought and they wouldn’t have figured this out without first trying the software… so there is constant evolution. Sam Newman’s book on Building Microservices recalls an interaction with a colleague Erik Doernenburg and how they conceptually think of architects more as town planners and not as maniacal micromanaging software purists.
- Need to manage the communication between the zones or regional contexts within our architecture, or this can easily be the demise to our systems… Need a coherent strategy to achieve this!
- Service-Oriented Architecture (SOA) is a design approach where multiple services collaborate together to provide some end set of capabilities. A service here typically means a completely separate operating system process. Communication between these services is done via calls across a network rather than method calls within a process boundary. SOA emerged as an approach to combat the challenges of the large monolithic applications. It is an approach which aims to promote the re-usability of software - two or more end-user applications for example could both use the same services. It is an approach which aims to make it easier to maintain or rewrite software, as theoretically we can replace one service with another without anyone knowing, as long as the semantics of the service don’t change too much. SOA at its heart is a very sensible idea. However, despite many efforts to the contrary, there is a lack of good consensus as how to do SOA well. In my opinion what much of the industry has failed to do is look holistically enough at the problem and present a compelling alternative to the narrative set out by various vendors in this space. Many of the problems lain at the door of SOA are actually problems with things like communication protocols (SOAP), vendor middle-ware, a lack of guidance about service granularity, or guidance on picking the wrong places to split your system. We’ll tackle each of these in turn throughout the rest of the book. A cynic might suggests that vendors co-opted (and in some cases drove) the SOA movement in a way to sell more products, and those self-same products in the end undermined the goal of SOA. Much of the conventional wisdom around SOA doesn’t help you understand how to split something big into something small. It doesn’t talk about how big is too big. It doesn’t talk enough about real-world practical ways to ensure that services do not become overly coupled. The number of things that go unsaid is where many of the pitfalls associated with SOA come from. The microservice approach has emerged from real-world use, taking our better understanding of systems and architecture on how to do SOA well. So you should instead think of Microservices as a specific approach for SOA in the same way that XP or Scrum are specific approaches for Agile software development.
- For many years now we have been finding better ways to build systems. We have been learning from what has come before, adopting new technology, and observing how a new wave of technology companies operate in different ways to create IT systems that help make both their customers and own developers happier. Eric Evans’ Domain Driven Design helped us understand the importance of representing the real world in our code, and showed us a path towards better ways to model our systems. Continuous Delivery showed how we can more effectively and efficiently get our software into production, instilling in us the idea that we should treat every check in as a release candidate. Our understanding of how the Web works has led us to develop better ways of having machines talk to machines. Alistar Cockburn’s Hexagonal Architecture guided us away from layered architectures where business logic could hide. Virtualisation platforms allowed us to provision and resize our machines at will, with Infrastructure Automation giving us a way to handle these machines at scale. Some large, successful organisations like Amazon and Google espoused the view of small teams owning the full life-cycle of their services. And more recently Netflix has shared with us ways of building anti-fragile systems at a scale that would be hard to comprehend just ten years before. Domain Driven Design. Continuous Delivery. On-demand virtualisation. Infrastructure automation. Small autonomous teams. Systems at Scale. Microservices have emerged from this world. They weren’t invented or described before the fact, they emerged as a trend, or a pattern, from real-world use. But they only exist because of all that has gone before. I will pull strands out of this prior work to help paint a picture of how to build, manage and evolve Microservices throughout the rest of this book. Many organisations have found that by embracing fine-grained, microservice architectures, they can deliver software faster and embrace newer technologies. Microservices give us significantly more freedom to allow us to react and make different decisions, allowing us to react faster to the inevitable change that impacts all of us.
- For many years now we have been finding better ways to build systems. We have been learning from what has come before, adopting new technology, and observing how a new wave of technology companies operate in different ways to create IT systems that help make both their customers and own developers happier. Eric Evans’ Domain Driven Design helped us understand the importance of representing the real world in our code, and showed us a path towards better ways to model our systems. Continuous Delivery showed how we can more effectively and efficiently get our software into production, instilling in us the idea that we should treat every check in as a release candidate. Our understanding of how the Web works has led us to develop better ways of having machines talk to machines. Alistar Cockburn’s Hexagonal Architecture guided us away from layered architectures where business logic could hide. Virtualisation platforms allowed us to provision and resize our machines at will, with Infrastructure Automation giving us a way to handle these machines at scale. Some large, successful organisations like Amazon and Google espoused the view of small teams owning the full life-cycle of their services. And more recently Netflix has shared with us ways of building anti-fragile systems at a scale that would be hard to comprehend just ten years before. Domain Driven Design. Continuous Delivery. On-demand virtualisation. Infrastructure automation. Small autonomous teams. Systems at Scale. Microservices have emerged from this world. They weren’t invented or described before the fact, they emerged as a trend, or a pattern, from real-world use. But they only exist because of all that has gone before. I will pull strands out of this prior work to help paint a picture of how to build, manage and evolve Microservices throughout the rest of this book. Many organisations have found that by embracing fine-grained, microservice architectures, they can deliver software faster and embrace newer technologies. Microservices give us significantly more freedom to allow us to react and make different decisions, allowing us to react faster to the inevitable change that impacts all of us.
- Intended for highly realiable, scalable distributed systems. Allow for planned/unplanned failures without compromising the entire system. You can know when a process dies and why it did (fabric?) You can force processes that depend on each other to die together if one of them goes wrong (co-locate systems in karaf) You can run a logger that automatically logs every uncaught exception for you, and even define your own Nodes can be monitored so you know when they went down (or got disconnected) You can restart failed processes (or groups of failed processes) Have whole applications restarting on different nodes if one fails And a lot more more stuff with the OTP framework
- For many years now we have been finding better ways to build systems. We have been learning from what has come before, adopting new technology, and observing how a new wave of technology companies operate in different ways to create IT systems that help make both their customers and own developers happier. Eric Evans’ Domain Driven Design helped us understand the importance of representing the real world in our code, and showed us a path towards better ways to model our systems. Continuous Delivery showed how we can more effectively and efficiently get our software into production, instilling in us the idea that we should treat every check in as a release candidate. Our understanding of how the Web works has led us to develop better ways of having machines talk to machines. Alistar Cockburn’s Hexagonal Architecture guided us away from layered architectures where business logic could hide. Virtualisation platforms allowed us to provision and resize our machines at will, with Infrastructure Automation giving us a way to handle these machines at scale. Some large, successful organisations like Amazon and Google espoused the view of small teams owning the full life-cycle of their services. And more recently Netflix has shared with us ways of building anti-fragile systems at a scale that would be hard to comprehend just ten years before. Domain Driven Design. Continuous Delivery. On-demand virtualisation. Infrastructure automation. Small autonomous teams. Systems at Scale. Microservices have emerged from this world. They weren’t invented or described before the fact, they emerged as a trend, or a pattern, from real-world use. But they only exist because of all that has gone before. I will pull strands out of this prior work to help paint a picture of how to build, manage and evolve Microservices throughout the rest of this book. Many organisations have found that by embracing fine-grained, microservice architectures, they can deliver software faster and embrace newer technologies. Microservices give us significantly more freedom to allow us to react and make different decisions, allowing us to react faster to the inevitable change that impacts all of us.
- Team communi – conways law…
- Communication is paramount!
- DSLs for Java, XML, OSGI, Groovy, Scala, Kotlin…highly polyglo
- Security Quality Freedom, no vendor lockin Interoperability Support Options Try before you buy http://www.pcworld.com/article/209891/10_reasons_open_source_is_good_for_business.html
- Integration is easy!!! Well… not really….