Confluent Platform 5.3 includes the following key updates: - Confluent Operator for Kubernetes and production-ready Ansible playbooks to automate deployment of Kafka and Confluent Platform on Kubernetes with cloud-native agility. - A redesigned user interface for Confluent Control Center and a new command line interface to better understand and manage event streams. - Role-based access control (in preview) and secret protection features to more securely access the platform. - Continued integration of the latest Apache Kafka 2.3 release with new connectors and incremental cooperative rebalancing.

1. 1
What’s New in
Confluent Platform 5.3
Kai Waehner
Technology Evangelist
contact@kai-waehner.de
LinkedIn
@KaiWaehner
www.confluent.io
www.kai-waehner.de
July 2019

2. 2
Confluent Platform
The Event Streaming Platform Built by the Original Creators of Apache Kafka®
Operations and Security
Development & Stream Processing
Apache Kafka
Confluent Platform
Support,Services,
Training,&Partners
Mission-Critical Reliability
Complete Event
Streaming Platform
Freedom of Choice
Datacenter Public Cloud Confluent Cloud
Self-Managed Software Fully Managed Service

3. 3
● Confluent Operator for
Kubernetes
● Production-Ready Ansible
Playbooks
What’s New in Confluent Platform 5.3
Automate with Cloud-
Native Agility
Secure Access to Your
Platform
Understand and Manage
Your Event Streams
Apache Kafka 2.3
● Redesigned Confluent
Control Center User
Interface
● New Command Line
Interface (CLI)
● Role-Based Access Control
(Preview)

4. 4
Confluent Platform Licensing
Open Source features
Apache Kafka®
Apache 2.0 License
Free. Unlimited Kafka brokers
Community support
Enterprise License (paid)
● Annual subscription
● 24x7 Confluent support
● Kafka Connect
● Kafka Streams
Apache ZooKeeper™
Clients
Ansible Playbooks
Community features
Connectors
Confluent Community License
Free. Unlimited Kafka brokers
Community support
REST Proxy
KSQL
Schema Registry
Commercial features
Connectors
Developer License
● Free
● Limited to 1 Kafka broker
● Community support
Evaluation License
● Free 30-day trial
● Unlimited Kafka brokers
● Community support
Control Center
Command Line Interface
Replicator
Auto Data Balancer
MQTT Proxy
Operator
Security Plugins
Role-Based Access Control (preview) ● Best-effort Confluent Support
New in CP 5.3

5. 55
Latest Version of Apache Kafka

6. 66
New features and bug fixes
● Complete overview:
○ https://www.confluent.io/blog/whats-new-in-apache-kafka-2-3
○ https://www.youtube.com/watch?v=sNqwJT2WguQ
● Highlight: Kafka Connect
● KIP-415: Incremental Cooperative Rebalancing in Kafka
Connect
● KIP-449: Add connector contexts to Connect worker logs
Apache Kafka
2.3
Always Built on the
Latest Version of
Apache Kafka

7. 7C O N F I D E N T I A L 7C O N F I D E N T I A L
Automate with Cloud-Native Agility

8. 88
What is Cloud-Native?
● Many Benefits compared to classical bare
metal / VM deployments
○ Scalable
○ Flexible
○ Agile
○ Elastic
○ Automated
● Some Definitions
○ https://12factor.net/
○ https://thenewstack.io/10-key-attributes-of-cloud-native-applications/
Modern Art of
Self-Managed
Software

9. 99
Why Confluent Operator?
● Kubernetes has become the open source
standard for orchestrating containerized
applications
● However, running stateful applications such
as Kafka can be very challenging and may
require a very specialized skill set to
accomplish manually
Automate with
Cloud-Native
Agility

10. 1010
Confluent Operator:
Apache Kafka on
Kubernetes made
simple
Run Apache Kafka and Confluent
Platform as a cloud-native application
on Kubernetes to minimize operating
complexity and increase developer
agility
Confluent Platform
Kubernetes
AWS Azure GCP
RH OpenShift Pivotal
On-Premises Cloud
Docker Images
Confluent Operator

11. 1111
Confluent
Operator
Deploy to Production in
Minutes
Automated deployment of
Confluent Platform resources:
Brokers, ZooKeeper, Kafka Connect,
KSQL, Schema Registry, Control
Center, and Replicator
Automate Key Lifecycle
Operations
● Failover
● Automated rolling upgrades
● Elastic scalability
Deploy on Any Platform,
On-Prem or in the Cloud
Run at Scale with
Confidence
Operationalizes years of Confluent
Cloud experience into a proven,
enterprise-grade solution that you
can deploy without deep Kafka
expertise
Deploy Apache Kafka
and Confluent Platform
as a cloud-native system
on Kubernetes
Kubernetes Engine Elastic Container
Service for Kubernetes
Kubernetes Service
https://www.slideshare.net/KaiWaehner
/confluent-operator-as-cloudnative-
kafka-operator-for-kubernetes

12. 1212
Why Ansible Playbooks?
● You need of simpler way to deploy
applications and infrastructure. Ansible is a
good tool to automate the process.
● Confluent offered open source Ansible
Playbooks for Confluent Platform
recommended for development only and
which were officially unsupported, making
them unviable in large-scale production
environments.
● https://github.com/confluentinc/cp-ansible
(Apache 2.0 License)
Automate with
Cloud-Native
Agility

13. 1313
New Production-Ready
Ansible Playbooks for
Confluent Platform
Simple and automated way to deploy
and configure the Confluent Platform
services:
● ZooKeeper
● Kafka Broker
● Kafka Connect
● KSQL
● Schema Registry
● REST Proxy
● Control Center
What’s New:
● Supported by Confluent
● Improved documentation
● Added support for CA-based TLS
certificates with two-way TLS mutual
authentication
● Added support for SASL GSSAPI
(Kerberos) for Kerberos authentication
● Added backward compatibility – deploy
two major versions backwards from the
latest major release

14. 14C O N F I D E N T I A L 14C O N F I D E N T I A L
Understand and Manage
Your Event Streams

15. 1515
Why a new Control Center UI?
● Control Center has introduced great
enhancements since Confluent Platform 5.0,
such as a consumer lag, message browser,
Schema Registry integration, KSQL UI,
dynamic broker configuration, multi-cluster
Connect and KSQL, increased scalability, and
more
● This time, we decided to improve the flow in
which you explore and manage Apache Kafka
Understand
and Manage
Your Event
Streams

16. 1616
Redesigned Control
Center User Interface
● Based on extensive customer
research and feedback
● Offers a more cohesive and
logical experience to build the
right mental model about the
platform
● Offers a consistent user
experience with Confluent Cloud

17. 17
At-a-glance view of key Apache Kafka metrics Improved message browser

18. 1818
Why a new CLI?
● Confluent has offered the Confluent CLI for
some time, but it was recommended for
development only
● To deliver an enterprise-ready platform, we
need a CLI that can support mission-critical
use cases
Understand
and Manage
Your Event
Streams

19. 1919
New Command Line
Interface
● Production-ready and fully
supported by Confluent
● Enables RBAC management
● Password protection
● Subsumed confluent-cli
commands for local
Manage your Confluent Platform.
Usage:
confluent [command]
Available Commands:
completion Print shell completion code.
help Help about any command
iam Manage RBAC and IAM permissions.
local Manage local Confluent Platform
development environment.
login Login to Confluent Platform.
logout Logout of Confluent Platform.
secret Manage secrets for Confluent Platform.
update Update the confluent CLI.
version Print the confluent CLI version.
Flags:
-h, --help help for confluent
-v, --verbose count Increase verbosity (-v for
warn, -vv for info, -vvv for debug, -vvvv for trace).
--version version for confluent

20. 20C O N F I D E N T I A L 20C O N F I D E N T I A L
Secure Access
to Your Platform

21. 2121
Why Role-Based Access
Control?
● As your usage of event streaming increases,
you may need to grant access to hundreds of
Confluent Platform users
● This will include not just Kafka but also Kafka
Connect, KSQL, Schema Registry, and more
Granularly
Access
to Your
Platform

22. 2222
Role-Based Access
Control (Preview)
What’s New:
● Configured via the new CLI
● Authorization enforced via
○ Control Center (GUI)
○ New CLI
○ APIs
● Enforced across all Confluent Platform
components:
○ KSQL, Connect, Schema Registry, REST
Proxy, and MQTT Proxy
● On Kafka Connect clusters, it provides
connector-level granularity
● Uses a set of seven predefined roles to
provide secure authorization of access to
resources by users and groups
Users/
Groups
Roles Resource
Scoping
CLI GUI API
Role
Binding
RBAC
Authorization

23. 23
Secret Protection
● Encrypts secrets within the configuration file
itself and does not expose the secrets in log
files
● Extends the security capabilities introduced
in KIP-226 for brokers and KIP-297 for
Connect to enable end-to-end secret
protection across all Confluent Platform
components
○ Kafka brokers, Connect, KSQL, Schema Registry,
Control Center, REST Proxy, etc.
Do not store secrets as
cleartext in files
Encrypt secrets
directly, so that they
are never stored in
cleartext

24. 2424
Confluent Platform
Download Options

25. 2525
Download
Confluent
Platform 5.3
Now
www.confluent.io/
download/

26. 2626
Confluent Cloud

27. 27
Best-of-breed Platforms, Partners and Services for Multi-cloud Streams
Private Cloud
Deploy on bare-metal, VMs,
containers or Kubernetes in your
datacenter with Confluent Platform
and Confluent Operator
Public Cloud
Implement self-managed in the public
cloud or adopt a fully managed service
with Confluent Cloud
Hybrid Cloud
Build a persistent bridge between
datacenter and cloud with
Confluent Replicator
Confluent
Replicator
VM
SELF MANAGED FULLY MANAGED

28. 28
Confluent Cloud
Cloud-Native Confluent Platform Fully-Managed Service
Available on the leading public clouds with mission-critical SLAs.
Serverless Kafka characteristics:
Pay-as-you-go, elastic auto-scaling, abstracting infrastructure (topics not brokers)

29. 29
Kafka Expertise, and Why it Matters?
Complexity
Large surface area
to cover
● Streaming systems are distributed
○ Many components with complex interactions
○ Challenging to optimize and troubleshoot
○ The Client also application matters
● Streaming systems are stateful
○ Capacity planning is non-trivial
○ Retention, memory, compute and n/w need sizing
● They require many APIs, metrics, systems, and configs
○ Difficult to secure and monitor
○ Time-consuming, difficult to learn and manage
DOWNTIME RISK
DATA LOSS
LATENCY
SECURITY RISK
TIME TO MARKET
SLOW TO LAUNCH
Non-trivial capacity
planning

30. 30
Confluent Cloud, What does Fully-managed Mean?
Infrastructure
management
(commodity)
Scaling
● Upgrades (latest stable version of Kafka)
● Patching
● Maintenance
● Sizing (retention, latency, throughput, storage, etc.)
● Data balancing for optimal performance
● Performance tuning for real-time and latency requirements
● Fixing Kafka bugs
● Uptime monitoring and proactive remediation of issues
● Recovery support from data corruption
● Scaling the cluster as needed
● Data balancing the cluster as nodes are added
● Support for any Kafka issue with less than 60 minute response time
Infra-as-a-Service
Harness full power of Kafka
Kafka-specific
management
Platform-as-a-Service
Evolve as you need
Future-proof
Mission-critical reliability
Most Kafka as a Service offerings are partially-managed

31. 3131
11. November 2019
Steigenberger Frankfurter Hof
13. November 2019
NOVOTEL Zürich City West
Ben Stopford
Office of the CTO
Confluent
Axel Löhn
Senior Project Manager
Deutsche Bahn
Kai Waehner,
Technologist
Confluent
Ralph Debusmann
IoT Solution Architect
Bosch Power Tools
cnfl.io/cse19frankfurt cnfl.io/cse19zurich

32. 32
Kai Waehner
Technology Evangelist
contact@kai-waehner.de
@KaiWaehner
www.kai-waehner.de
www.confluent.io
LinkedIn
Questions? Feedback?
Let’s connect!