Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Best Practices for Middleware and Integration Architecture Modernization with Apache Camel

62 views

Published on

What are important considerations when modernizing middleware and moving towards serverless and/or cloud native integration architectures? How can we make the most of flexible technologies such as Camel K, Kafka, Quarkus and OpenShift. Claus is working as project lead on Apache Camel and has extensive experience from open source product development.

The talk was recorded and runs for 30 minutes and published on youtube at: https://www.youtube.com/watch?v=d1Hr78a7Lww

Published in: Software
  • Be the first to like this

Best Practices for Middleware and Integration Architecture Modernization with Apache Camel

  1. 1. Best Practices for Middleware and Integration Architecture Modernization Claus Ibsen Red Hat March 2020 With Apache Camel
  2. 2. About Claus Ibsen ● Senior Principal Software Engineer at Red Hat ● ASF Member & Java Champion ● Apache Camel Tech Lead ● Author of Camel in Action books ● Based in Denmark ● Blog: https://medium.com/@davsclaus ● Linkedin: https://www.linkedin.com/in/davsclaus ● Twitter: @davsclaus
  3. 3. Agenda ● What’s Camel ● Camel 3 ● Integration Evolution ● Serverless ● Camel K ● Camel Quarkus ● Camel Kafka Connector ● More Material
  4. 4. What’s Camel
  5. 5. What is Apache Camel ● The swizz army of integration ● 12+ years of development and one of the most active Apache projects
  6. 6. What is Apache Camel ● Packed with functionality The giant Swiss army knife 😅
  7. 7. System Integration
  8. 8. System Integration
  9. 9. What is Apache Camel ● Java based integration framework ● Runtime support: Spring Boot, Quarks, Java EE, Microprofile, OSGi, Standalone, etc. ● Based on Enterprise Integration Patterns ● Comes with 300+ components (connectors) ● DSL (Java or XML) to describe integration flow (routes) ● Can integrate everything ... almost everything
  10. 10. Camel Routes from("file:data/inbox") .to("jms:queue:order"); <route> <from uri="file:data/inbox"/> <to uri="jms:queue:order"/> </route> Java DSL XML DSL
  11. 11. Apache Camel 3
  12. 12. Apache Camel 3 - Projects Camel Swiss knife of integration
  13. 13. Apache Camel 3 - Projects Camel Swiss knife of integration Camel Spring Boot Camel on Spring Boot
  14. 14. Apache Camel 3 - Projects Camel Swiss knife of integration Camel Spring Boot Camel on Spring Boot Camel Karaf Camel on Apache Karaf / OSGi
  15. 15. Apache Camel 3 - Projects Camel K Camel on Kubernetes & Knative Camel Swiss knife of integration Camel Spring Boot Camel on Spring Boot Camel Karaf Camel on Apache Karaf / OSGi
  16. 16. Apache Camel 3 - Projects Camel Quarkus Optimized JVM & Native compiled Java (GraalVM) Camel K Camel on Kubernetes & Knative Camel Swiss knife of integration Camel Spring Boot Camel on Spring Boot Camel Karaf Camel on Apache Karaf / OSGi
  17. 17. Apache Camel 3 - Projects Camel Quarkus Optimized JVM & Native compiled Java (GraalVM) Camel K Camel on Kubernetes & Knative Camel Swiss knife of integration Camel Spring Boot Camel on Spring Boot Camel Karaf Camel on Apache Karaf / OSGi Camel Kafka Connector Kafka Connector with Camel
  18. 18. Integration Evolution
  19. 19. Integration 10 years ago
  20. 20. Integration Architectures
  21. 21. Very old SOA/ESB Architecture
  22. 22. Today's ESB Architecture Apache Camel OSGi Framework Apache CXF Apache ActiveMQ App App App App App App App App App App App App Deployer Shell Config Manage Blueprint HTTP Integration Core Application Apache Karaf
  23. 23. Integration Architectures
  24. 24. Microservice Architecture (MSA 1.0) Data store SaaS
  25. 25. Today's Microservice Architecture (MSA 1.0) Apache Camel Apache Tomcat Spring Rest App Spring Framework Spring Boot Integration Core Application Spring Boot Hystrix Circuit Breaker
  26. 26. Integration Architectures
  27. 27. Cloud Native Architecture Camel App Integration Core Application Service Mesh Pod Sidecar Camel Spring Boot Spring Boot Tomcat
  28. 28. Cloud Native Architecture Camel App Integration Core Application Service Mesh Pod Sidecar Camel Main
  29. 29. Cloud Native Architecture Camel App Integration Application Service Mesh Pod Sidecar Camel Quarkus Quarkus Core
  30. 30. Serverless Architecture Camel App Integration Core Application Knative Pod Sidecar Camel K K
  31. 31. Serverless Requirements For the platform ● Scaling to zero ● Rapid scaling up and down ● Eventing mechanism ● Routing and networking For the application runtime ● Fast startup ● Fast first response ● Low on memory and CPU ● Small size on disk
  32. 32. Serverless Requirements For the platform ● Scaling to zero ● Rapid scaling up and down ● Eventing mechanism ● Routing and networking For the application runtime ● Fast startup ● Fast first response ● Low on memory and CPU ● Small size on disk K
  33. 33. Camel K
  34. 34. A lightweight integration platform, based on Apache Camel, born on Kubernetes, with serverless superpowers. What is Apache Camel K ?
  35. 35. A lightweight integration platform, based on Apache Camel, born on Kubernetes, with serverless superpowers. What is Apache Camel K ? Runs on “vanilla” Kubernetes (1) ... K 1.
  36. 36. A lightweight integration platform, based on Apache Camel, born on Kubernetes, with serverless superpowers. What is Apache Camel K ? Runs on “vanilla” Kubernetes (1), Openshift (2) ... K 1. K 2.
  37. 37. A lightweight integration platform, based on Apache Camel, born on Kubernetes, with serverless superpowers. What is Apache Camel K ? Runs on “vanilla” Kubernetes (1), Openshift (2) and gives its best on a Knative-powered cluster (3)! K 1. K 2. K 3.
  38. 38. How?
  39. 39. How? from(“telegram:bots/bot-id”) .transform()... .to(“kafka:topic”); from(“kafka:topic”) .to(“http:my-host/api/path”); 1. Create a integration file (Java, Groovy, Kotlin, JS, XML…) Camel DSL, based on EIPs...
  40. 40. How? from(“telegram:bots/bot-id”) .transform()... .to(“kafka:topic”); from(“kafka:topic”) .to(“http:my-host/api/path”); 1. Create a integration file (Java, Groovy, Kotlin, JS, XML…) Camel DSL, based on EIPs... $ kamel run integration.groovy 2. Run it
  41. 41. How? from(“telegram:bots/bot-id”) .transform()... .to(“kafka:topic”); from(“kafka:topic”) .to(“http:my-host/api/path”); 1. Create a integration file (Java, Groovy, Kotlin, JS, XML…) Camel DSL, based on EIPs... 3. Runs on Openshift or Kubernetes $ kamel run integration.groovy 2. Run it
  42. 42. Architecture of Camel K Dev Environment Remote Cloud kamel CLI Camel K Operator “Integration” Custom Resource Running Pod Live updates! Fast redeploy! Less than 1 second! Tailored for cloud-native development experience
  43. 43. Fast Deployment of Camel K Time to run a integration using different strategies (in seconds) Lower is better 😃 F-m-p is the “fabric8-maven-plugin” (http://maven.fabric8.io/) deploying a average spring-boot based integration on Minishift vs. a remote OpenShift cluster (accounting time to upload the fat Jar). Source S2I build has been measured in Red Hat Fuse Online.
  44. 44. Camel K on OperatorHub
  45. 45. Quick Camel K Demo
  46. 46. Camel Quarkus
  47. 47. Java Density Problem CONTAINER ORCHESTRATION Node Node Node Traditional Cloud-Native Java Stack Traditional Cloud-Native Java Stack Traditional Cloud-Native Java Stack Traditional Cloud-Native Java Stack NodeJS NodeJS NodeJS NodeJS NodeJS NodeJS NodeJS Go Go Go Go Go Go Go Go Go Go Go Go Go Go Go Go Go Go Go Go Go https://developers.redhat.com/blog/2017/03/14/java-inside-docker/
  48. 48. SCALE-UP SCALE-DOWN SCALE-TO- ZERO Java Fast Scaling Problem
  49. 49. What is Quarkus ?
  50. 50. What is Quarkus ? Supersonic Subatomic Java
  51. 51. What is Quarkus ? Supersonic Subatomic Java A Kubernetes Native Java stack tailored for GraalVM & OpenJDK HotSpot, crafted from the best of breed Java libraries and standards
  52. 52. Minimal Footprint
  53. 53. Apache Camel and Quarkus runs blazingly fast with native compiled binaries. Startup is 13 msec, 68mb binary file, and 16mb RSS memory used.
  54. 54. Camel Quarkus Demos
  55. 55. Camel Kafka Connector
  56. 56. What is Apache Kafka? ● Kafka is known as ○ a distributed streaming platform ○ or a pub/sub messaging broker ● It’s an ecosystem ○ Multiple components part of Apache Kafka ○ A lot of 3rd party integrations ○ One of them is Kafka Connect Streams API Producer API Consumer API 3rd party tools Mirror Maker Connect
  57. 57. What is Kafka Connect? ● It is a framework that helps you to integrate Kafka with other systems ○ A user can define source and sink connectors to stream data in/out of Kafka brokers ○ Connectors are plugable - you can use one of the many connectors available or write your own ● Distributed and scalable by default ● Automatic offset management ● Simple transformations ● Streaming / batch integration Kafka Connect: Source Connector Kafka Connect: Sink Connector External System External System
  58. 58. What is Camel Kafka Connector? ● A Kafka Connector built on top of Apache Camel ● Started as an internal proof-of-concept ○ A sub-project of the Apache Camel ○ Donated by Red Hat to the ASF on December 2019 ● Reuses in a simple way most of the Camel components as Kafka sink and sources
  59. 59. Available Connectors Amazon AWS Kinesis Cassandra CQL JMS Amazon AWS S3 ElasticSearch Telegram Amazon AWS SNS File Syslog Amazon AWS SQS HTTP
  60. 60. Demo S3 to JMS ● https://github.com/oscerd/camel-kafka-connector-demo ● S3 Source connector (camel-aws-s3) ● JMS Sink connector (camel-sjms2) ● A file loaded on a bucket -> file content in a JMS Queue To run Kafka on Kubernetes
  61. 61. No coding Only configuration
  62. 62. More Material
  63. 63. More Camel Material ● Apache Camel K https://github.com/apache/camel-k ● Apache Camel Quarkus https://github.com/apache/camel-quarkus ● Apache Camel Kafka Connector https://github.com/apache/camel-kafka-connector ● Quick Camel K demo https://www.youtube.com/watch?v=I-g2xt-Qcb8 ● Camel K Introduction Blog https://www.nicolaferraro.me/2018/10/15/introducing-camel-k ● Kubernetes enterprise integration patterns with Camel K webinar https://www.youtube.com/watch?v=51x9BewGCYA ● Camel K and Knative video https://www.youtube.com/watch?v=lOqubmVSGdw
  64. 64. More Material ● Webinar - Apache Camel 3 is here what’s new https://www.redhat.com/en/events/webinar/apache-camel-3-here-whats-new ● QCon 2020 Talk by Bilgin - Evolution of Distributed Systems on Kubernetes https://www.infoq.com/presentations/kubernetes-primitives-design-patterns https://k8spatterns.io/ https://www.manning.com/books/camel-in-action-second-edition
  65. 65. ● Red Hat Integration https://www.redhat.com/en/products/integration ● Red Hat Middleware https://developers.redhat.com/middleware ● Red Hat Event Driven Architecture https://developers.redhat.com/topics/event-driven More Material
  66. 66. Any Questions ? Follow us on Twitter @davsclaus @ApacheCamel https://github.com/apache/camel A star on github is appreciated

×