Cloud-native data infrastructure, such as Confluent and Kubernetes, combine well together to enable teams to use declarative spec based automation (GitOps) for deployment and management. In this talk, we'll showcase how to use GitOps to manage data in motion with Confluent for Kubernetes.

1. GitOps with Confluent and
Kubernetes
Rohit Bakhshi, Confluent
Kafka Summit 2021

2. What is GitOps?
Continuous Deployment for cloud native applications
● Declarative Specs (Infrastructure as Code) in Git
● Automation to deploy and maintain to match the declarative spec

3. An API-driven Control Plane to deploy and manage Confluent
● Declarative API (CRDs) for Confluent Platform Services
● Declarative API (CRDs) for topics and RBAC policies
Integrates with Cloud-Native ecosystem for Security, Reliability, DevOps Automation
● Complete automated secure setup
● Secrets Management with Kubernetes Secrets and Hashicorp Vault
● One command scale up
● Fault tolerance to zone and node failure
Introducing Confluent for Kubernetes

4. An API-driven Control Plane to deploy and manage Confluent
● Declarative API (CRDs) for Confluent Platform Services
● Declarative API (CRDs) for topics and RBAC policies
Integrates with Cloud-Native ecosystem for Security, Reliability, DevOps Automation
● Complete automated secure setup
● Secrets Management with Kubernetes Secrets and Hashicorp Vault
● One command scale up
● Fault tolerance to zone and node failure
Introducing Confluent Operator 2.0

5. Confluent + Kubernetes => GitOps
5
Confluent Operator
Component
(Custom Resource)
Credentials
kafka.yaml
---
kafka:
spec:
listeners:
internal:
authentication:
type: plain
jaasConfig:
secretRef: credential
Helm
K8s Secrets
K8s API

6. Continuous Deployment on Kubernetes
Flux CD
Operator
Git Repo
Kafka CR
Zookeeper CR
ControlCenter CR
Confluent
Operator
KafkaTopic CRs
/cluster/prod
/cluster/qa

7. LDAP
Demo
QA
- SASL/Plain authentication
- TLS encryption with automatically
generated certs
Prod
- SASL/Plain authentication
- TLS encryption with custom certs
- Granular Role Based Access Control
Control Center
ZooKeeper x 3
Confluent Operator
Kubernetes
Kafka Broker x 3
Schema Registry
Connect
KsqlDB

8. Try it and Learn More!
Try Confluent for
Kubernetes
cnfl.io/download
Confluent Blog
cnfl.io/blog
Community
cnfl.io/meetups