Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Serverless integration with Knative and Apache Camel on Kubernetes

197 views

Published on

This presentation will introduce Knative, an open source project that adds serverless capabilities on top of Kubernetes, and present Camel K, a lightweight platform that brings Apache Camel integrations in the serverless world. Camel K allows running Camel routes on top of any Kubernetes cluster, leveraging Knative serverless capabilities such as “scaling to zero”.

We will demo how Camel K can connect cloud services or enterprise applications using its 250+ components and how it can intelligently route events within the Knative environment via enterprise integration patterns (EIP).

Target Group: Developers, architects and other technical people - a basic understanding of Kubernetes is an advantage

Published in: Software
  • Be the first to comment

Serverless integration with Knative and Apache Camel on Kubernetes

  1. 1. Serverless Integration with Knative & Apache Camel on Kubernetes Claus Ibsen @davsclaus
  2. 2. About me ● Senior Principal Software Engineer at Red Hat ● Java Champion ● 11 years as Apache Camel committer ● Author of Camel in Action books 🌍 http://www.davsclaus.com @davsclaus davsclaus ✉️ claus.ibsen@gmail.com
  3. 3. ● The swiss knife of integration ● >10 years of development - still one of the most active Apache projects ● Java-based integration framework that can be installed on multiple runtimes: Spring-Boot, JEE, JBoss, Karaf, Standalone, ... ● Based on Enterprise Integration Patterns ● Supports 250+ components ● Uses a powerful DSL ● Can integrate anything http://camel.apache.org What is Apache Camel?
  4. 4. +
  5. 5. + +
  6. 6. + + +
  7. 7. + + + =
  8. 8. ● A platform for directly running integrations on Openshift and Kubernetes ● Based on (ex Core OS) operator-sdk ● A community-driven project ● A subproject of Apache Camel started on August 31st, 2018 What is Apache Camel K? https://github.com/apache/camel-k K
  9. 9. Overview of Camel K A lightweight platform for running Camel integration DSL in the cloud. ● Based on operator-sdk ● Works on Openshift and Kubernetes $ kamel run routes.groovy from(“telegram:bots/bot-id”) .transform()... .to(“kafka:topic”); from(“kafka:topic”) .to(“http:my-host/api/path”); File routes.groovy
  10. 10. Architecture of Camel K Dev Environment Remote Cloud kamel CLI Camel K Operator “Integration” Custom Resource Running Pod Live updates! Fast redeploy! Less than 1 second! Tailored for cloud-native development experience
  11. 11. Camel K in details kind: Integration apiVersion: camel.apache.org/v1alpha1 metadata: name: my-integration spec: sources: - name: source.groovy content: |- from(“telegram:...”) .transform()... .to(“kafka:...”) from(“telegram:bots/bot-id”) .transform()... .to(“kafka:topic”); from(“kafka:topic”) .to(“http:my-host/api/path”); This is what people care about: Camel DSL Camel K Operator 1. Choose a runtime 2. Create container image 3. Create Kubernetes resources for deployment kamel CLI
  12. 12. Camel K Resources Visualization $ kamel run code.groovyfrom(“telegram:bots/bot-id”) .transform()... .to(“kafka:topic”); from(“kafka:topic”) .to(“http:my-host/api/path”); File code.groovy <<custom-resource>> IntegrationPlatform <<custom-resource>> IntegrationContext <<custom-resource>> Integration from(“telegram:..”) .to(“...”) BuildConfig ImageStream ImageStreamTag Deployment ConfigMap from(“telegram:..”) .to(“...”) Pod On OpenShift. It uses Kaniko on “vanilla” Kubernetes. CRDs allow to abstract from all this implementation details!
  13. 13. Camel K Resources Visualization $ kamel run code.groovyfrom(“telegram:bots/bot-id”) .transform()... .to(“kafka:topic”); from(“kafka:topic”) .to(“http:my-host/api/path”); File code.groovy <<custom-resource>> IntegrationPlatform <<custom-resource>> IntegrationContext <<custom-resource>> Integration from(“telegram:..”) .to(“...”) BuildConfig ImageStream ImageStreamTag Deployment ConfigMap from(“telegram:..”) .to(“...”) Pod On OpenShift. It uses Kaniko on “vanilla” Kubernetes.
  14. 14. Fast Deployment of Camel K Time to run a integration using different strategies (in seconds) Lower is better 😃 F-m-p is the “fabric8-maven-plugin” (http://maven.fabric8.io/) deploying a average spring-boot based integration on Minishift vs. a remote OpenShift cluster (accounting time to upload the fat Jar). Source S2I build has been measured in Red Hat Fuse Online.
  15. 15. Can you implement Serverless without functions ?
  16. 16. “Serverless computing refers to the concept of building and running applications that do not require server management. It describes a finer-grained deployment model where applications, bundled as one or more functions, are uploaded to a platform and then executed, scaled, and billed in response to the exact demand needed at the moment” Source: https://www.cncf.io/blog/2018/02/14/cncf-takes-first-step-towards-serverless-computing/ What is serverless again… ?
  17. 17. What is Knative ?
  18. 18. Build A pluggable model for building artifacts, like jar files, zips or containers from source code. Knative Overview - Components Serving An event-driven model that serves the container with your application and can "scale to zero". Eventing Common infrastructure for consuming and producing events that will stimulate applications. "...an extension to Kubernetes exposing building blocks to build modern, source-centric, and container- based applications that can run anywhere".
  19. 19. Knative Overview - Builds ● A Build is a list of containers run in-order, with source mounted in (TaskRun) ● BuildTemplates provide reusable, parameterized recipes that can be used to create Builds (Task) ● Pipelines - The future! ○ Group of Tasks ● Build Example ● "Source to URL" ● S2I for OpenShift users apiVersion: build.knative.dev/v1alpha1 kind: Build metadata: name: example-build spec: serviceAccountName: build-auth-example source: git: url: https://github.com/example/build-example.git revision: master steps: - name: centos-example image: centos args: ["centos-build-example", "SECRETS-example.md"] steps: - image: quay.io/example-builders/build-example args: ['echo', 'hello-example', 'build']
  20. 20. Knative Overview - Serving ● Configurations represent the ‘floating HEAD’ of a history of Revisions ● Revisions represent immutable snapshot of code and configuration ● Routes configure ingress over a collection of Revisions and/or Configurations ● Services (nope, not K8s services) are top- level controllers that manage a set of Routes and Configurations to implement a network service Service (my-function) Route (name) Configuration Revision Revision Revision manages records history of 10% 90%
  21. 21. Knative Overview - Eventing ● Goal is to be a generalized eventing framework ● Many influencers also active on CloudEvents specification Current-ish API: ● Receive Adapter triggers on EventSources that send Events to a sink (channel) ● Channels are named endpoints which accept event delivery (provided by Kafka, AMQP…) ● Subscriptions register to receive traffic from a Channel
  22. 22. Knative Eventing - Pipeline Knative Eventing Docs
  23. 23. Knative Eventing - Flow Knative Eventing Docs
  24. 24. Again, beyond functions... Keep code and configuration separate Each configuration change triggers the creation of new revision Concurrency (scale out via process) Disposability (fast startup/graceful shutdown) Build, release, run (separate build and run stages) https://12factor.net/ Configuration(configuration.serving.knative.dev) Revision(revision.serving.knative.dev) Route(route.serving.knative.dev) Service(service.serving.knative.dev) Build (buildtemplate.build.knative.dev)
  25. 25. What about Camel K & Knative ? K
  26. 26. Camel K and Knative <<custom-resource>> Build Knative building blocks relevance to Camel <<custom-resource>> ...<<custom-resource>> ...<<custom-resource>> ... Knative Build <<custom-resource>> Service <<custom-resource>> ...<<custom-resource>> ...<<custom-resource>> ... Knative Serving <<custom-resource>> ...<<custom-resource>> ...<<custom-resource>> ... Knative Eventing Standardize building container images Auto-scaling and scale-to-zero Messaging for event-based applications <<custom-resource>> Channel X
  27. 27. Camel K and Knative <<custom-resource>> Build Knative building blocks relevance to Camel <<custom-resource>> ...<<custom-resource>> ...<<custom-resource>> ... Knative Build <<custom-resource>> Service <<custom-resource>> ...<<custom-resource>> ...<<custom-resource>> ... Knative Serving <<custom-resource>> ...<<custom-resource>> ...<<custom-resource>> ... Knative Eventing Standardize building container images Auto-scaling and scale-to-zero Messaging for event-based applications <<custom-resource>> Channel X
  28. 28. Camel K and Knative <<custom-resource>> Build Knative building blocks relevance to Camel <<custom-resource>> ...<<custom-resource>> ...<<custom-resource>> ... Knative Build <<custom-resource>> Service <<custom-resource>> ...<<custom-resource>> ...<<custom-resource>> ... Knative Serving <<custom-resource>> ...<<custom-resource>> ...<<custom-resource>> ... Knative Eventing Standardize building container images Auto-scaling and scale-to-zero Messaging for event-based applications <<custom-resource>> Channel X
  29. 29. Camel & Knative in details kind: Integration apiVersion: camel.apache.org/v1alpha1 metadata: name: my-integration spec: sources: - name: source.groovy content: |- from(“knative:channel/a”) .to(“http:my-host/api/path”) from(“knative:channel/a”) .to(“http:my-host/api/path”); Camel K Operator Knative profile? kind: Service apiVersion: serving.knative.dev/v1alpha1 yes kind: Deployment # standard one no
  30. 30. Camel & Knative in details $ kamel run code.groovy from(“knative:channel/a”) .transform()... .to(“slack:#room”); File code.groovy <<custom-resource>> Integration from(“knative:channel/a”) .to(“...”) <<custom-resource>> Service <<custom-resource>> Channel <<custom-resource>> Subscription Pod Pod Pod events Same user experience! Or even no pods! In the knative profile, the operator uses Knative building blocks instead of a standard Kubernetes “Deployment”
  31. 31. Camel K: same model for different purposes Camel K & Knative Eventing Channel Channel Channel Ext System Event Source EIP Integration Function 250+ Camel components!
  32. 32. ● Apache Camel K https://github.com/apache/camel-k ● Camel K Introduction Blog https://www.nicolaferraro.me/2018/10/15/introducing-camel-k ● Camel K and Knative tutorial https://redhat-developer-demos.github.io/knative-tutorial/knative- tutorial-camelk/v1.0.0/index.html ● Camel K and Knative video https://www.youtube.com/watch?v=btf_e2GniXM ● Camel Knative Event Sources https://github.com/knative/eventing- sources/tree/master/contrib/camel/samples Camel K & Knative links
  33. 33. Java Density Problem CONTAINER ORCHESTRATION Node Node Node Traditional Cloud-Native Java Stack Traditional Cloud-Native Java Stack Traditional Cloud-Native Java Stack Traditional Cloud-Native Java Stack NodeJS NodeJS NodeJS NodeJS NodeJS NodeJS NodeJS Go Go Go Go Go Go Go Go Go Go Go Go Go Go Go Go Go Go Go Go Go https://developers.redhat.com/blog/2017/03/14/java-inside-docker/
  34. 34. SCALE-UP SCALE-DOWN SCALE-TO- ZERO Java Fast Scaling Problem
  35. 35. Supersonic Subatomic Java https://quarkus.io
  36. 36. What is Quarkus A Kubernetes Native Java stack tailored for GraalVM & OpenJDK HotSpot, crafted from the best of breed Java libraries and standards
  37. 37. Minimal Footprint - Quarkus ● Minimal footprint Java applications ○ Native vs Quarkus+OpenJDK JVM vs Traditional JVM
  38. 38. Container First - Quakrus ● First Class Support for Graal/SubstrateVM ○ Native compiled first-class supported ● Build Time Metadata Processing ○ Pre-Build and "warump" as much as possible during build time ● Reduction in Reflection Usage ○ Reduce startup and memory ● Native Image Pre Boot ○ Most of startup serialized into native image to startup even faster https://quarkus.io/vision/container-first
  39. 39. Apache Camel and Quarkus runs blazingly fast 🚀 with native compiled binaries. Startup is 7 msec, 28mb binary file, and 15mb RSS memory used.
  40. 40. Camel and Quarkus - Links ● Quarkus https://quarkus.io ● Quarkus Camel Extensions https://quarkus.io/extensions/#integration https://github.com/quarkusio/quarkus/tree/master/extensions/camel ● Quarkus Camel Demo https://github.com/quarkusio/quarkus-quickstarts/tree/master/camel-java
  41. 41. Where are the slides ? https://www.slideshare.net/davsclaus
  42. 42. Any Questions ? 🌍 http://www.davsclaus.com @davsclaus davsclaus ✉️ claus.ibsen@gmail.com

×