In this session, we will explore key challenges with function interactions and coordination, addressing these problems using Enterprise Integration Patterns (EIP) and modern approaches with the latest innovations from the Apache Camel community:
Apache Camel is the Swiss army knife of integration, and the most powerful integration framework. In this session you will hear about the latest features in the brand new 3rd generation.
Camel K, is a lightweight integration platform that enables Enterprise Integration Patterns to be used natively on any Kubernetes cluster. When used in combination with Knative, a framework that adds serverless building blocks to Kubernetes, and the subatomic execution environment of Quarkus, Camel K can mix serverless features such as auto-scaling, scaling to zero, and event-based communication with the outstanding integration capabilities of Apache Camel.
- Apache Camel 3
- Camel K
- Camel Quarkus
We will show how Camel K works. We’ll also use examples to demonstrate how Camel K makes it easier to connect to cloud services or enterprise applications using some of the 300 components that Camel provides.
Unveiling the Tech Salsa of LAMs with Janus in Real-Time Applications
Apache Camel v3, Camel K and Camel Quarkus
1. Apache Camel v3
Camel K
Camel Quarkus
Claus Ibsen
Senior Principal Software Engineer
Red Hat
November 2019
2. About me
Claus Ibsen
● Senior Principal Software Engineer at Red Hat
● ASF Member & Java Champion
● Apache Camel tech lead
● Author of Camel in Action books
● Based in Denmark
● Blog: https://medium.com/@davsclaus
3. Agenda
● Integration Evolution
● Apache Camel
● Camel 3
● Camel K
● Serverless
● Camel K & Knative
● Camel Quarkus
● Roadmap
● More Material
18. What is Apache Camel
● The swiss knife of integration
● 10+ years of development and one
of the most active Apache projects
19. What is Apache Camel
● Packed with functionality
The giant Swiss knife 😅
20. What is Apache Camel
● Java based integration framework
● Runtime support: Spring Boot, Apache Karaf / OSGi, JEE / WAR,
Microprofile, Quarkus, Standalone, and more ...
● Based on Enterprise Integration Patterns
● Comes with 300+ components (connectors)
● DSL (Java or XML) to describe integration flow (routes)
● Can integrate everything ... almost everything
● 10+ years of development - still one of the most active Apache projects
● The cornerstone of Red Hat Fuse
58. Camel 3 - Reactive
● Reactive Core
○ All EIPs now fully reactive
○ Blocking vs Non-Blocking code-paths merged into one (Non-Blocking)
● Postponed for Camel 3.x
○ Backpressure
○ Client API
○ Java 9+ Flowable API (requires to drop Java 8)
59. Camel 3 - Camel Main
● Standalone Camel
○ Bootstrap for Camel K & Camel Quarkus
○ Unified configuration (also for Camel Spring Boot)
● Configurable
○ application.properties (ala Spring Boot)
● Tooling Support
○ Eclipse
○ Eclipse Che
○ VSCode
○ IDEA
61. Camel 3 - Other
● JUnit 5 support
○ camel-test-junit5
● Step EIP
○ Group together N+ EIPs (logical name)
○ Grouped performance statistics
● Lazy start producer
○ Avoid Camel to fail on startup if producer fails to start
(typically connecting to remote system)
● Writeable Registry
○ Easier to add your own beans
● Route Model more extensible
○ YAML route builder (Camel K)
62. 19/12/2018
Master switched
to camel 3
02/2019
M1
❏ New Camel API
❏ Move components
out of camel core
❏ Extract basic
classes in a support
JAR
❏ Reactive Routing
Engine
❏ Faster releases
❏ Quarkus Support
for a subset of
components
❏ Writeable registry
❏ Step EIP
❏ Faster build
❏ Reactive Routing
Engine improved
❏ Java 11 support
improved
❏ EIP at scale
❏ More Quarkus
support
Final
Release
04/2019
M2
06/2019
M3
07/2019
M4
❏ Auto-generate
Endpoint DSL
❏ Camel-core-engine
❏ Camel-main
improved
❏ Website and docs
almost done
09/2019
RC1
❏ Renaming
components
❏ New website live
❏ Stabilizing
❏ Reduce footprint
Camel 3 - Timeline
10/2019
RC2 RC3
12/2019
GA
5700+ commits (Camel 3.x)
64. A lightweight integration platform, based on Apache Camel,
born on Kubernetes, with serverless superpowers.
What is Apache Camel K ?
65. A lightweight integration platform, based on Apache Camel,
born on Kubernetes, with serverless superpowers.
What is Apache Camel K ?
Runs on “vanilla” Kubernetes (1) ...
K
1.
66. A lightweight integration platform, based on Apache Camel,
born on Kubernetes, with serverless superpowers.
What is Apache Camel K ?
Runs on “vanilla” Kubernetes (1), Openshift (2) ...
K
1.
K
2.
67. A lightweight integration platform, based on Apache Camel,
born on Kubernetes, with serverless superpowers.
What is Apache Camel K ?
Runs on “vanilla” Kubernetes (1), Openshift (2) and gives its best on a Knative-powered cluster (3)!
K
1.
K
2.
K
3.
72. Architecture of Camel K
Dev Environment Remote Cloud
kamel CLI
Camel K
Operator
“Integration”
Custom
Resource
Running Pod
Live updates!
Fast redeploy!
Less than 1 second!
Tailored for cloud-native development experience
73. Fast Deployment of Camel K
Time to run a integration using different strategies (in seconds)
Lower is better
😃
F-m-p is the “fabric8-maven-plugin”
(http://maven.fabric8.io/) deploying a
average spring-boot based
integration on Minishift vs. a remote
OpenShift cluster (accounting time to
upload the fat Jar). Source S2I build
has been measured in Red Hat Fuse
Online.
79. What is serverless?
“Serverless computing refers to the concept of building and running applications
that don’t require server management. It describes a finer-grained deployment
model where applications, bundled as one or more functions, are uploaded to a
platform and executed, scaled, billed in response to the exact demand needed”
https://github.com/cncf/wg-serverless
Serverless personas:
● Developer: writes code for, and benefits from the serverless platform.
● Provider: manages the serverless platform.
80. What is serverless?
“Serverless computing refers to the concept of building and running applications that
don’t require server management. It describes a finer-grained deployment model
where applications, bundled as one or more functions, are uploaded to a platform and
executed, scaled, billed in response to the exact demand needed”
https://github.com/cncf/wg-serverless
Serverless personas:
● Developer: writes code for, and benefits from the serverless platform.
● Provider: manages the serverless platform.
81. Serverless (Compute)
Here’s my personal definition of “serverless”.
Before it becomes a billing model, we need to define it “technically”.
Serverless is a programming model based on:
82. Serverless (Compute)
Here’s my personal definition of “serverless”.
Before it becomes a billing model, we need to define it “technically”.
Serverless is a programming model based on:
...
1) Auto-scaling services/functions
(including scaling to zero)
83. 2) Event-driven execution
Serverless (Compute)
Here’s my personal definition of “serverless”.
Before it becomes a billing model, we need to define it “technically”.
Serverless is a programming model based on:
...
1) Auto-scaling services/functions
(including scaling to zero)
84. Serverless requirements
For the platform
● Scaling to zero
● Rapid scaling up and down
● Eventing mechanism
● Routing and networking
For the application runtime
● Fast startup
● Fast first response
● Low on memory and CPU
● Small size on disk
85. Serverless requirements
For the platform
● Scaling to zero
● Rapid scaling up and down
● Eventing mechanism
● Routing and networking
For the application runtime
● Fast startup
● Fast first response
● Low on memory and CPU
● Small size on disk
K
88. Build
A pluggable model for
building artifacts, like jar
files, zips or containers
from source code.
Knative Overview - Components
Serving
An event-driven model
that serves the container
with your application and
can "scale to zero".
Eventing
Common infrastructure for
consuming and producing
events that will stimulate
applications.
Kubernetes-based platform to build, deploy, and manage modern serverless workloads
90. Camel K and Knative
<<custom-resource>>
Build
Knative components relevance to Camel
<<custom-resource>>
...<<custom-resource>>
...<<custom-resource>>
...
Knative Build
<<custom-resource>>
Service
<<custom-resource>>
...<<custom-resource>>
...<<custom-resource>>
...
Knative Serving
<<custom-resource>>
...<<custom-resource>>
...<<custom-resource>>
...
Knative Eventing
Standardize building
container images
Auto-scaling and scale-to-zero Messaging for event-based
applications
<<custom-resource>>
Channel
91. Camel K and Knative
<<custom-resource>>
Build
Knative components relevance to Camel
<<custom-resource>>
...<<custom-resource>>
...<<custom-resource>>
...
Knative Build
<<custom-resource>>
Service
<<custom-resource>>
...<<custom-resource>>
...<<custom-resource>>
...
Knative Serving
<<custom-resource>>
...<<custom-resource>>
...<<custom-resource>>
...
Knative Eventing
Standardize building
container images
Auto-scaling and scale-to-zero Messaging for event-based
applications
<<custom-resource>>
Channel
XNo longer included in
Knative
Use external projects
such as Tekton
92. Camel K and Knative
<<custom-resource>>
Build
Knative components relevance to Camel
<<custom-resource>>
...<<custom-resource>>
...<<custom-resource>>
...
Knative Build
<<custom-resource>>
Service
<<custom-resource>>
...<<custom-resource>>
...<<custom-resource>>
...
Knative Serving
<<custom-resource>>
...<<custom-resource>>
...<<custom-resource>>
...
Knative Eventing
Standardize building
container images
Auto-scaling and scale-to-zero Messaging for event-based
applications
<<custom-resource>>
Channel
X
93. Camel K and Knative Serving
Kubernetes Namespace
rest().post(“/path”)
.to(“xx:system1”)
.to(“xx:system2”)
300+ components!
System 1
System 2
Knative
Service
94. Camel K and Knative Serving
Kubernetes Namespace
rest().post(“/path”)
.to(“xx:system1”)
.to(“xx:system2”)
300+ components!
System 1
System 2
Knative
Service
95. Camel K and Knative Serving
Kubernetes Namespace
rest().post(“/path”)
.to(“xx:system1”)
.to(“xx:system2”)
300+ components!
System 1
System 2
Knative
Service
Request
96. Camel K and Knative Serving
Kubernetes Namespace
A container is created only when needed!
rest().post(“/path”)
.to(“xx:system1”)
.to(“xx:system2”)
300+ components!
System 1
System 2
Knative
Service
Request Pod
97. Camel K and Knative Serving
Kubernetes Namespace
rest().post(“/path”)
.to(“xx:system1”)
.to(“xx:system2”)
300+ components!
System 1
System 2
Knative
Service
Request Pod
98. Camel K and Knative Serving
Kubernetes Namespace
rest().post(“/path”)
.to(“xx:system1”)
.to(“xx:system2”)
300+ components!
System 1
System 2
Knative
Service
Request Pod
99. Camel K and Knative Serving
Kubernetes Namespace
Scale to zero when no load
rest().post(“/path”)
.to(“xx:system1”)
.to(“xx:system2”)
300+ components!
System 1
System 2
Knative
Service
100. Camel K and Knative Serving
Kubernetes Namespace
rest().post(“/path”)
.to(“xx:system1”)
.to(“xx:system2”)
300+ components!
System 1
System 2
Knative
Service
Multiple containers under high load!
Request Pod
Request
Request
Pod
Pod
101. Camel K and Knative Serving
Kubernetes Namespace
rest().post(“/path”)
.to(“xx:system1”)
.to(“xx:system2”)
300+ components!
System 1
System 2
Knative
Service
Scale to zero when no load
102. Camel K and Knative
<<custom-resource>>
Build
Knative components relevance to Camel
<<custom-resource>>
...<<custom-resource>>
...<<custom-resource>>
...
Knative Build
<<custom-resource>>
Service
<<custom-resource>>
...<<custom-resource>>
...<<custom-resource>>
...
Knative Serving
<<custom-resource>>
...<<custom-resource>>
...<<custom-resource>>
...
Knative Eventing
Standardize building
container images
Auto-scaling and scale-to-zero Messaging for event-based
applications
<<custom-resource>>
Channel
X
103.
104. Kubernetes Namespace
Camel K and Knative Eventing
300+ components!
System 1
System 2
Channel a
Persisted by
Kafka, in-memory
(pluggable)
105. Kubernetes Namespace
Camel K and Knative Eventing
from(“knative:channel/a”)
.to(“xx:system1”)
.to(“xx:system2”)
300+ components!
System 1
System 2
Knative
Service
Knative
Subscription
Channel a
106. Kubernetes Namespace
Camel K and Knative Eventing
from(“knative:channel/a”)
.to(“xx:system1”)
.to(“xx:system2”)
300+ components!
System 1
System 2
Knative
Service
Knative
Subscription
Cloud Event
https://cloudevents.io/
Channel a
107. Kubernetes Namespace
Camel K and Knative Eventing
from(“knative:channel/a”)
.to(“xx:system1”)
.to(“xx:system2”)
300+ components!
System 1
System 2
Knative
Service
Knative
Subscription
PodCloud Event
https://cloudevents.io/
Channel a
108. Kubernetes Namespace
Camel K and Knative Eventing
from(“knative:channel/a”)
.to(“xx:system1”)
.to(“xx:system2”)
300+ components!
System 1
System 2
Knative
Service
Knative
Subscription
PodCloud Event
https://cloudevents.io/
Channel a
109. Camel K: same model for different purposes
Camel K and Knative Eventing
Channel
Channel
Channel
Ext
System
110. Camel K: same model for different purposes
Camel K and Knative Eventing
Channel
Channel
Channel
Ext
System
Event
Source
(Sink)
300+ components!
111. Camel K: same model for different purposes
Camel K and Knative Eventing
Channel
Channel
Channel
Ext
System
Event
Source
(Sink)
Enterprise
Integration
Patterns
(Routing)
300+ components!
65+ patterns
112. Camel K: same model for different purposes
Camel K and Knative Eventing
Channel
Channel
Channel
Ext
System
Event
Source
(Sink)
Enterprise
Integration
Patterns
(Routing)
Function
(Integration)
300+ components!
300+ components!
65+ patterns
113. Knative Eventing includes Camel
Standard Knative
Sources
That can be used
without any knowledge
of Camel K!
... but Camel K is under the
hood :)
114. Apache Camel K on CNCF
https://camel.apache.org/blog/CamelK-CNCF-Landscape/
119. Camel K and Knative Demo
legacy.
groovy
channel/in
120. Camel K and Knative Demo
legacy.
groovy
channel/in
process.
groovy
121. Camel K and Knative Demo
legacy.
groovy
channel/in
channel/out
channel/null
process.
groovy
122. Camel K and Knative Demo
legacy.
groovy
channel/in
channel/out
channel/null
process.
groovy
quarkus.
yaml
123. Camel K and Knative Demo
legacy.
groovy
channel/in
channel/out
channel/null
process.
groovy
quarkus.
yaml
telegram.
groovy
124. Camel K and Knative Demo
legacy.
groovy
channel/in
channel/out
channel/null
process.
groovy
quarkus.
yaml
telegram.
groovy
125. Camel K and Knative Demo
legacy.
groovy
channel/in
channel/out
channel/null
process.
groovy
quarkus.
yaml
telegram.
groovy
126.
127. Camel K and Knative Demo
legacy.
groovy
channel/in
channel/out
channel/null
process.
groovy
quarkus.
yaml
telegram.
groovy
128.
129. Camel K and Knative Demo
legacy.
groovy
channel/in
channel/out
channel/null
process.
groovy
quarkus.
yaml
telegram.
groovy
130.
131. Camel K and Knative Demo
legacy.
groovy
channel/in
channel/out
channel/null
process.
groovy
quarkus.
yaml
telegram.
groovy
132. Quarkus is now supported out of the box with
Camel K since this demo was created
(no longer needed to build your own kit)
https://camel.apache.org/camel-k/latest/traits/quarkus.html
133. Camel K and Knative Demo
legacy.
groovy
channel/in
channel/out
channel/null
process.
groovy
quarkus.
yaml
telegram.
groovy
134.
135. Camel K and Knative Demo
legacy.
groovy
channel/in
channel/out
channel/null
process.
groovy
quarkus.
yaml
telegram.
groovy
139. Characteristic Monolithic/SOA Microservies/CNA Functions/Serverless
Packaging Ear/War/Jar Fat-jar Native executable
Startup procedure XML scanning Annotation scanning Instant (ahead-of-time compilation)
Memory/binary size 1000s MBs 100s MBs 10s of MBs
Startup/first response Minutes Seconds Milliseconds
Process lifecycle Long running Managed by platform Ephemeral (executed per request)
State (app/user) Stateful Stateless (no user state) Ephemeral (no application state)
Data ownership Shared data model Owns data No data ownership
Operations Multiple unrelated operations Multiple domain operations Single event driven operation
Scope Multiple business domains Single business domain Single task
Motivation Non-scalable, high complexity
domains
Portable, autonomous,
scalable workloads
Rapid productivity, at the expense
of control and portability
Application trends (Java)
142. Java Density Problem
CONTAINER ORCHESTRATION
Node Node Node
Traditional Cloud-Native
Java Stack
Traditional Cloud-Native
Java Stack
Traditional Cloud-Native
Java Stack
Traditional Cloud-Native
Java Stack
NodeJS
NodeJS
NodeJS
NodeJS
NodeJS
NodeJS
NodeJS
Go Go Go
Go Go Go
Go Go Go
Go Go Go
Go Go Go
Go Go Go
Go Go Go
https://developers.redhat.com/blog/2017/03/14/java-inside-docker/
146. What is Quarkus ?
Supersonic Subatomic Java
A Kubernetes Native Java stack tailored for GraalVM &
OpenJDK HotSpot, crafted from the best of breed Java libraries
and standards
152. Camel K - Roadmap
● Support for Camel 3 (Now RC3, GA in Nov/Dec 2019)
○ Lightweight modules to reduce footprint
○ Webhook meta-component (more components that
can scale down to 0)
○ Camel K 1.0.0-M3 supports Camel 3.0.0-RC1
● Quarkus as integration runtime
○ Reduced footprint and cold start-up time
○ Native image build in parallel swapped with JVM
image during auto-scaling once ready
● Knative 1.0 → Camel K 1.0.0
● Support the Broker / Trigger model
● Support Tekton builds
● Push-based endpoints
● Fully integrated with Syndesis / Fuse Online
● Better IDE support
● Better integration with CI tools
● BDD testing
● Operator Capability Level 3+
153. Camel Quarkus - Roadmap
● Camel Extensions on http://code.quarkus.io
● Port more Camel components as Extensions
(22 done out of 300)
● More build time optimisations
● Reduce startup overhead
● Quarkus 1.0 → Camel Quarkus 1.0.0
● Better IDE support
154. Camel Kafka Connector
● A kafka connector source and sink implementation that embeds camel.
● It is configurable to use most of the camel-components.
● Making it able to talk with 300+ different kind of external systems.
300+ components!
155. Camel Debezium
● Stream changes from databases to Camel
● camel-debezium component in Camel 3
○ MySQL connector
○ Other databases on the way ...
Debezium
https://www.youtube.com/watch?time_continue=12&v=xLXx8bUyTtk
161. Serverless Integration
Microservices Microservices Microservices External SaaS
External SaaS
External SaaS
External SaaS
Integration
function
Integration
function
Integration
function
Integration
function
Integration
function
KNative
Service
Service
Service
Service
Microservices Microservices
function
166. More Material
● Apache Camel K
https://github.com/apache/camel-k
● Apache Camel Quarkus
https://github.com/apache/camel-quarkus
● Quick Camel K demo
https://www.youtube.com/watch?v=I-g2xt-Qcb8
● Camel K Introduction Blog
https://www.nicolaferraro.me/2018/10/15/introducing-camel-k
● Kubernetes enterprise integration patterns with Camel K webinar
https://www.youtube.com/watch?v=51x9BewGCYA
● Camel K and Knative video
https://www.youtube.com/watch?v=lOqubmVSGdw