1
Fundamentals for Apache Kafka®
Benefits of Stream Processing and Apache Kafka Use Cases
Mark Fei, Sr. Technical Trainer, Confluent
2
Session Schedule
● Session 1: Benefits of Stream Processing and Apache Kafka Use Cases
● Session 2: Apache Kafka Architecture & Fundamentals Explained
● Session 3: How Apache Kafka Works
● Session 4: Integrating Apache Kafka into your Environment
3
Learning Objectives
After this module you will be able to:
● Identify the need for event-driven systems
● Name specific use cases where real-time data is
critical
● Explain the advantages of using Apache Kafka and
Confluent Platform as your Event Streaming platform
4
Motivation
The shift to Event-driven Systems has Already Begun …
5
Motivation
This leads us to …
A single platform to connect everyone to every event
Real-time streams of events
All events stored for historical view
6
Motivation
Successful Digital Businesses are Inherently Event-driven
Born cloud-native … Traditional ones that adapt …
Social Networks
Enabling Event
Sharing
Streaming Provider
On-demand Digital
Content
Newspaper
Provide a single
Source of Truth
Credit Card Payments
Microservices
Architecture
7
Motivation
Born cloud-native … Traditional ones that adapt …
Ride Hailing
Connecting
Provider with
Customer in real-
time
Connected Cars
IoT – Real-time
Traffic Routing
8
Motivation
Apache Kafka: The De-
facto Standard for Real-
Time Event Streaming
• Global-scale
• Real-time
• Persistent Storage
• Stream Processing
9
Motivation
Thousands of Companies Worldwide trust Kafka for their Journey
towards “Event-driven”
https://kafka.apache.org/powered-by
10
Motivation
Over 35% of Fortune
500 Companies
Already Trust Kafka
for Mission Critical
Apps
11
Real-time Fraud Detection
● Act in real-time
● Detect Fraud
● Minimize risk
● Improve customer experience
12
Automotive
The Future of the
Automotive Industry is
a Real Time Data
Cluster
13
Real-time e-Commerce
Rewards Program
● Onboarding new merchants faster
● Increased speed at which mobile
applications are delivered to customers
● Enabled a full 360 view of customers
● Enhanced performance and monitoring
● Projected savings of millions of dollars
14
Customer 360
● Improved data integration
● Increased up-sell and cross-sell
opportunities
● Increased scalability and flexibility
● Saved costs
15
Core Banking
● Empowered successful core banking platform
relaunch
● Met HA and SLA needs
● Improved scalability
● Power AI for ChatBots
16
Online Gaming
Data Pipelining
● Increased reliability
● Accurate, real-time data
● Ability to process data at scale
● Faster ramp time
17
Government
● Near real-time events and better data
quality
● Increased efficiency
● Ability to change their organization
● Produce & store population data from
several sources
● Reduce welfare crime through strengthened
identity management
● Provide better privacy and meet GDPR
requirements
18
Financial Services
Customer communications
● Enhanced the customer experience
● Enabled “One Bank” strategy
Payments engine
● Improved fraud detection engine,
saving millions of euros
● Grew topics in production by 600
percent
1919
Confluent Platform
20
Central Nervous System
21
The Maturity Model
Kafka is a Good Starting Point, Confluent Completes the Journey
22
Confluent Platform
Build a Central Nervous
System for your Modern
Event-driven Enterprise
23
Confluent Platform
The Confluent Platform
enables companies to
respond accurately and in
real-time to business
events
24
Confluent Platform
Confluent is Enabling Event-driven Transformation across Industries
25
Confluent Platform
A Complete Set of Development, Operations, and Management Capabilities to run
Kafka at Scale
26
Confluent Platform
27
Confluent Platform Deployment Models
28
Confluent Cloud
29
Confluent Control Center
Kafka is powerful … but has many parts
30
Confluent Control Center
Management and Monitoring for the Enterprise
31
Confluent CLI
● Platforms: Linux, Unix-based
● License: Proprietary
● Packaging: Independent of CP
● Key Features:
○ RBAC management
○ Password protection
○ Subsumed confluent-cli
commands for local
32
Role Based Access Control
● Availability: CP 5.3
Utilizes a predefined set of Roles
● User who is assigned a role receives all
privileges of that role
● Each user could belong to multiple roles
● Privileged user within a given scope can
create/update other user’s roles
● Privileged user can update roles for existing
users
● Across the platform (KSQL, Connect,
Schema Registry, Connectors, etc.)
33
Confluent Operator
Confluent Operator for
Kubernetes runs on any
platform at any scale
What:
Deployment and management
automation for Confluent
Platform across environments
For Whom:
Organizations that have
standardized on Kubernetes as
the platform runtime
36
Confluent – Feature Comparison
Feature Confluent Platform (On Premise) Confluent Cloud (Fully Managed)
Scale Unlimited throughput,
unlimited retention
Unlimited throughput,
unlimited retention (CCE)
Availability Self Managed 99.95% uptime SLA
Durability Self Managed Multi-AZ with 3 availability zones
(option)
Updates 3-4 Annual Releases Weekly Fully Managed Updates
Support 24x7 Gold and Platinum (option) 24x7 Gold SVPC peering (option)
Pricing Node based pricing Usage Based Pricing
Packaging/Environment Tarball, deb, rpm, zip,
Docker images
AWS, GCP, Azure
37
Broad Connector Eco-System
38
Confluent Hub
Discover and use
trusted sources and
sinks from Confluent,
our partners, and the
community.
39
Confluent Hub
GOAL: Connect Everything!
40
Continue your Apache Kafka Education!
● Confluent Operations for Apache Kafka
● Confluent Developer Skills for Building Apache Kafka
● Confluent Stream Processing using Apache Kafka
Streams and KSQL
● Confluent Advanced Skills for Optimizing Apache Kafka
For more details, seehttp://confluent.io/training
4141
Certifications
Confluent Certified Developer
for Apache Kafka
(aligns to Confluent Developer Skills
for Building Apache Kafka course)
Confluent Certified
Administrator for Apache
Kafka
(aligns to Confluent Operations Skills
for Apache Kafka)
What you Need to Know
○ Qualifications: 6-to-9 months hands-on
experience
○ Duration: 90 mins
○ Availability: Live, online 24/7
○ Cost: $150
○ Register online:
www.confluent.io/certification
4242
cnfl.io/slack
Stay in touch!
cnfl.io/kafka-trainingcnfl.io/download
4343
cnfl.io/blog cnfl.io/slack
Stay in touch!
cnfl.io/kafka-training
44
Thank you for attending!
• Thank you for attending thesession!
• Feedback to: training-admin@confluent.io
45
Copyright ©Confluent, Inc. 2014-2019. Privacy Policy | Terms &Conditions.
Apache, Apache Kafka, Kafka and the Kafka logo are trademarks of
the Apache Software Foundation

Benefits of Stream Processing and Apache Kafka Use Cases

  • 1.
    1 Fundamentals for ApacheKafka® Benefits of Stream Processing and Apache Kafka Use Cases Mark Fei, Sr. Technical Trainer, Confluent
  • 2.
    2 Session Schedule ● Session1: Benefits of Stream Processing and Apache Kafka Use Cases ● Session 2: Apache Kafka Architecture & Fundamentals Explained ● Session 3: How Apache Kafka Works ● Session 4: Integrating Apache Kafka into your Environment
  • 3.
    3 Learning Objectives After thismodule you will be able to: ● Identify the need for event-driven systems ● Name specific use cases where real-time data is critical ● Explain the advantages of using Apache Kafka and Confluent Platform as your Event Streaming platform
  • 4.
    4 Motivation The shift toEvent-driven Systems has Already Begun …
  • 5.
    5 Motivation This leads usto … A single platform to connect everyone to every event Real-time streams of events All events stored for historical view
  • 6.
    6 Motivation Successful Digital Businessesare Inherently Event-driven Born cloud-native … Traditional ones that adapt … Social Networks Enabling Event Sharing Streaming Provider On-demand Digital Content Newspaper Provide a single Source of Truth Credit Card Payments Microservices Architecture
  • 7.
    7 Motivation Born cloud-native …Traditional ones that adapt … Ride Hailing Connecting Provider with Customer in real- time Connected Cars IoT – Real-time Traffic Routing
  • 8.
    8 Motivation Apache Kafka: TheDe- facto Standard for Real- Time Event Streaming • Global-scale • Real-time • Persistent Storage • Stream Processing
  • 9.
    9 Motivation Thousands of CompaniesWorldwide trust Kafka for their Journey towards “Event-driven” https://kafka.apache.org/powered-by
  • 10.
    10 Motivation Over 35% ofFortune 500 Companies Already Trust Kafka for Mission Critical Apps
  • 11.
    11 Real-time Fraud Detection ●Act in real-time ● Detect Fraud ● Minimize risk ● Improve customer experience
  • 12.
    12 Automotive The Future ofthe Automotive Industry is a Real Time Data Cluster
  • 13.
    13 Real-time e-Commerce Rewards Program ●Onboarding new merchants faster ● Increased speed at which mobile applications are delivered to customers ● Enabled a full 360 view of customers ● Enhanced performance and monitoring ● Projected savings of millions of dollars
  • 14.
    14 Customer 360 ● Improveddata integration ● Increased up-sell and cross-sell opportunities ● Increased scalability and flexibility ● Saved costs
  • 15.
    15 Core Banking ● Empoweredsuccessful core banking platform relaunch ● Met HA and SLA needs ● Improved scalability ● Power AI for ChatBots
  • 16.
    16 Online Gaming Data Pipelining ●Increased reliability ● Accurate, real-time data ● Ability to process data at scale ● Faster ramp time
  • 17.
    17 Government ● Near real-timeevents and better data quality ● Increased efficiency ● Ability to change their organization ● Produce & store population data from several sources ● Reduce welfare crime through strengthened identity management ● Provide better privacy and meet GDPR requirements
  • 18.
    18 Financial Services Customer communications ●Enhanced the customer experience ● Enabled “One Bank” strategy Payments engine ● Improved fraud detection engine, saving millions of euros ● Grew topics in production by 600 percent
  • 19.
  • 20.
  • 21.
    21 The Maturity Model Kafkais a Good Starting Point, Confluent Completes the Journey
  • 22.
    22 Confluent Platform Build aCentral Nervous System for your Modern Event-driven Enterprise
  • 23.
    23 Confluent Platform The ConfluentPlatform enables companies to respond accurately and in real-time to business events
  • 24.
    24 Confluent Platform Confluent isEnabling Event-driven Transformation across Industries
  • 25.
    25 Confluent Platform A CompleteSet of Development, Operations, and Management Capabilities to run Kafka at Scale
  • 26.
  • 27.
  • 28.
  • 29.
    29 Confluent Control Center Kafkais powerful … but has many parts
  • 30.
    30 Confluent Control Center Managementand Monitoring for the Enterprise
  • 31.
    31 Confluent CLI ● Platforms:Linux, Unix-based ● License: Proprietary ● Packaging: Independent of CP ● Key Features: ○ RBAC management ○ Password protection ○ Subsumed confluent-cli commands for local
  • 32.
    32 Role Based AccessControl ● Availability: CP 5.3 Utilizes a predefined set of Roles ● User who is assigned a role receives all privileges of that role ● Each user could belong to multiple roles ● Privileged user within a given scope can create/update other user’s roles ● Privileged user can update roles for existing users ● Across the platform (KSQL, Connect, Schema Registry, Connectors, etc.)
  • 33.
    33 Confluent Operator Confluent Operatorfor Kubernetes runs on any platform at any scale What: Deployment and management automation for Confluent Platform across environments For Whom: Organizations that have standardized on Kubernetes as the platform runtime
  • 34.
    36 Confluent – FeatureComparison Feature Confluent Platform (On Premise) Confluent Cloud (Fully Managed) Scale Unlimited throughput, unlimited retention Unlimited throughput, unlimited retention (CCE) Availability Self Managed 99.95% uptime SLA Durability Self Managed Multi-AZ with 3 availability zones (option) Updates 3-4 Annual Releases Weekly Fully Managed Updates Support 24x7 Gold and Platinum (option) 24x7 Gold SVPC peering (option) Pricing Node based pricing Usage Based Pricing Packaging/Environment Tarball, deb, rpm, zip, Docker images AWS, GCP, Azure
  • 35.
  • 36.
    38 Confluent Hub Discover anduse trusted sources and sinks from Confluent, our partners, and the community.
  • 37.
  • 38.
    40 Continue your ApacheKafka Education! ● Confluent Operations for Apache Kafka ● Confluent Developer Skills for Building Apache Kafka ● Confluent Stream Processing using Apache Kafka Streams and KSQL ● Confluent Advanced Skills for Optimizing Apache Kafka For more details, seehttp://confluent.io/training
  • 39.
    4141 Certifications Confluent Certified Developer forApache Kafka (aligns to Confluent Developer Skills for Building Apache Kafka course) Confluent Certified Administrator for Apache Kafka (aligns to Confluent Operations Skills for Apache Kafka) What you Need to Know ○ Qualifications: 6-to-9 months hands-on experience ○ Duration: 90 mins ○ Availability: Live, online 24/7 ○ Cost: $150 ○ Register online: www.confluent.io/certification
  • 40.
  • 41.
    4343 cnfl.io/blog cnfl.io/slack Stay intouch! cnfl.io/kafka-training
  • 42.
    44 Thank you forattending! • Thank you for attending thesession! • Feedback to: training-admin@confluent.io
  • 43.
    45 Copyright ©Confluent, Inc.2014-2019. Privacy Policy | Terms &Conditions. Apache, Apache Kafka, Kafka and the Kafka logo are trademarks of the Apache Software Foundation