Applying ML on your Data in Motion with AWS and Confluent | Joseph ...

Ad

Ad

Ad

Ad

Ad

Ad

Ad

Ad

Ad

Ad

Ad

Ad

Applying ML on your Data in
Motion with AWS and Conﬂuent

Joseph Morais
AWS Evangelist/Cloud Partner Solutions Architect, Confluent
theejosephmorais
Kanchan Waikar
Senior Partner S...

Joseph Morais
AWS Evangelist/Cloud Partner Solutions Architect, Confluent
theejosephmorais

Data in Motion
And why you might care

Event Streaming is the
Central Nervous System
for today’s enterprises.
Apache Kafka®
is the technology.

‘Event’ is what happens in your business
Transportation
TPMS sensor in Carol’s car detected low tire-pressure at 5:11am.
K...

The Rise of Event Streaming
80%
Fortune 100 Companies
Using Apache Kafka
(majority are Conﬂuent customers)

Event Streaming allow us to set Data in Motion:
Continuously processing evolving streams of data in real-time
Rich front-e...

At Conﬂuent,
streaming is in our DNA.
We help the world’s
largest organizations
make it part of theirs.

Modernize
your apps
DATA INTEGRATION
Connected car
Fraud detection
Customer 360
Personalized
promotions
Apps driven by
rea...

Lock-in avoidance
and community
development
Enterprises want to avoid
the high costs of
yesteryear’s lock-in as
they moder...

What or who is Kafka?
The append-only log

Both Kafkas like to write
Time

Writers
Kafka
cluster
Readers

Guarantees of a Database
• Strict ordering
• Persistence

Rewind & Replay
Rewind & Replay
Reset to any point in the shared
narrative

Distributed by design
• Replication
• Fault Tolerance
• Partitioning
• Elastic Scaling

Producing to Kafka
Time

Kafka Topics
my-topic
my-topic-partition-0
my-topic-partition-1
my-topic-partition-2
broker-1
broker-2
broker-3

Partition Leadership and Replication
Broker 1
Topic1
partition1
Broker 2 Broker 3 Broker 4
Topic1
partition1
Topic1
partit...

Partition Leadership and Replication - node failure
Broker 1
Topic1
partition1
Broker 2 Broker 3 Broker 4
Topic1
partition...

Kafka Connect and Kafka Streams
Sink
Source
KAFKA
STREAMS
KAFKA
CONNECT
KAFKA
CONNECT
Your App

Kafka Connect: Reliable and scalable integration
of Kafka with other systems
• Centralized management and conﬁguration
• S...

Kafka Streams: Write standard Java apps and
microservices to process your data in real-time
• No separate processing clust...

Stream processing with Kafka
Example: Using Kafka’s Streams API for writing
elastic, scalable, fault-tolerant Java and Sca...

ksqlDB to the rescue

Conﬂuent makes Kafka Easier
But how though?

https://www.conﬂuent.io/hub/
Large Ecosystem for Event Streaming
Easily connect to 130+ data systems
Data Diode
Amazon Red...

Confluent: Everywhere
Confluent Platform
The Enterprise Distribution of
Apache Kafka
Confluent Cloud
Apache Kafka Re-engineer...

ksqlDB at a Glance
What is it?
ksqlDB is an event streaming
database for working with
streams and tables of data.
All the ...

Built on the Best Technology,
Available as a Fully-Managed Service
Kafka is the backbone of ksqlDB
ksqlDB is built on top ...

Federated streaming, hybrid
and multi-cloud.
Data syndication and replication
across and between clouds and
on-premises, w...

Conﬂuent: Complete
Dynamic Performance & Elasticity
Self-Balancing Clusters | Tiered Storage
Flexible DevOps Automation
Op...

Architecture Example
Cloud-native
Services
...
Customer services & apps
on-premises (EU)
Customer services & apps
on-premi...

Large Partner Network
Consulting Partners, Cloud Partners, OEM Partners, Tech Partners
and more
https://www.conﬂuent.io/pa...

What makes Conﬂuent unique?
Everywhere
Global availability on AWS,
or on-prem
Bridge on-prem to cloud
with cluster linking...

Ingest & Process
Capture event streams with a consistent data structure using
Schema Registry, develop real-time ETL pipel...

Serverless integration
Connect existing and apps & data stores in a repeatable way without
having to manage- Apache Kafka,...

Accelerate modernization from on-prem to AWS
Redshift Sink
Lambda Sink
AWS Direct
Connect
Replicator
LEGACY EDW
MAINFRAME
...

Challenge
Maximize customer satisfaction and revenue growth by creating a
hyper-personalized online retail experience, tur...

Kanchan Waikar
Senior Partner Solutions Architect at AWS
kanchanwaikar

Share Slideshare

LinkedIn
Facebook
Twitter

Embed

Size (px)

Show related Slideshows at end

WordPress Shortcode

Link

Share
Email

2024 Trend Updates: What Really Wor... by Search Engine Jou... 1011447 views
Storytelling For The Web: Integrate... by Chiara Aliotta 919244 views
Artificial Intelligence, Data and C... by OECD Directorate ... 820562 views
How to Leverage AI to Boost Employe... by SocialHRCamp 342424 views
2024 State of Marketing Report – by... by Marius Sescu 195380 views
Everything You Need To Know About C... by Expeed Software 218594 views

View on Slideshare

1 of 43 Ad

View on Slideshare

1 of 43 Ad