This document discusses building modern streaming analytics architectures with Confluent on AWS. It outlines challenges with traditional analytics approaches, and how a modern data strategy using real-time streaming analytics addresses those challenges. It describes how to build seamless streaming architectures using services like Amazon Kinesis and Confluent Kafka, integrated through AWS and Confluent. Examples of real-time analytics use cases are also provided.

1. © 2023, Amazon Web Services, Inc. or its affiliates.
© 2023, Amazon Web Services, Inc. or its affiliates.
Nae Ittaboon (he/him)
Solutions Architect
Amazon Web Services (AWS)
Building Modern Streaming Analytics
with Confluent on AWS
#DataInMotionTour

2. © 2023, Amazon Web Services, Inc. or its affiliates.
© 2023, Amazon Web Services, Inc. or its affiliates.
Agenda
2
❑ Develop a modern data strategy
❑ Act on events in real time with data streaming
❑ Build seamless streaming with AWS and Confluent
❑ Innovate together to power customer success
❑ Key takeaways

3. © 2023, Amazon Web Services, Inc. or its affiliates.
Challenges of data analytics at scale
3
Data volume and velocity
Multiple analytics needs
Variety of sources and data types
Difficult to manage systems
Complex to scale
Inflexible tools
Security, compliance
Slow performance Increasing and unpredictable cost

4. © 2023, Amazon Web Services, Inc. or its affiliates.
Catalog
Governance
Data
sources
People,
apps, and
devices
Analytics
Machine
learning
Databases
Data
lakes
Modern data strategy

5. © 2023, Amazon Web Services, Inc. or its affiliates.
Amazon
OpenSearch
Service
Amazon
Aurora
Amazon
EMR
Amazon
SageMaker
Amazon
DynamoDB
Amazon
Redshift
Amazon S3
Modern data architecture on AWS
Key Pillars
Data at any scale
Seamless data access and movement
Purpose-built for best price performance
Built-in ML to solve business challenges
Unified governance

6. © 2023, Amazon Web Services, Inc. or its affiliates.
The benefits of data lakes
Data lake
Catalog
Store all your data in open formats
Decouple storage from compute
Cost-effectively scale storage to exabytes
Process data in place
Choice of analytical and ML engines

7. © 2023, Amazon Web Services, Inc. or its affiliates.
Sharing across data movement with Data Mesh
Data producers Data mesh Data consumers
Unique modern data architecture
Suited to business function
Teams that want to share data
Unique modern data architecture
Suited to business function
Team that runs the marketplace Teams that want to use data

8. © 2023, Amazon Web Services, Inc. or its affiliates.
Source: Perishable insights, Mike Gualtieri, Forrester
Real time Seconds Minutes Hours Days Months
Value
of
data
to
decision-making
Preventive/predictive
Actionable Reactive Historical
Time-critical decisions Traditional “batch” business intelligence
Information half-life
in decision-making
Why real-time data streaming analytics ?
Data loses value quickly over time

9. © 2023, Amazon Web Services, Inc. or its affiliates.
Common real-time analytics use cases
Anomaly and fraud detection
Empowering IoT analytics
Nourishing marketing campaigns
Real-time personalization
Tailoring customer experience in real time
Supporting healthcare and emergency services

10. © 2023, Amazon Web Services, Inc. or its affiliates.
Devices and/or
applications
that produce
real-time
data at high
velocity
Data from tens of
thousands of data
sources can be written to
a single stream
Data are stored in the
order they were
received for a set
duration
of time and can be
replayed indefinitely
during that time
Records are read in
the order they are
produced, enabling real-
time analytics or
streaming ETL
Database (NoSQL
most common)
Data lake
Data warehouse
Event driven
Applications
`
Analytics
dashboard
Real-time streaming analytics pipeline
Ingest, Process & Analyze High Volumes of High-Velocity Data from Various Sources in Real Time

11. © 2023, Amazon Web Services, Inc. or its affiliates.
Mobile device
Metering
Click streams
IoT sensors
AWS SDKs
Kinesis Agent/KPL
`
Apache Kafka
Amazon Kinesis Data
Streams
Amazon Kinesis Data
Firehose
Apache Kafka
Amazon Kinesis Data
Streams
Amazon Kinesis
Data Analytics
AWS Glue Streaming
Amazon EMR
Amazon DynamoDB
Amazon EMR
Amazon S3
Amazon Redshift Amazon OpenSearch
Amazon QuickSight
Apache Kafka
Streaming analytics on AWS

12. © 2023, Amazon Web Services, Inc. or its affiliates.
Rich front-end
customer
experiences
Real-time
Event Streams and Analysis
A Sale A shipment
A Trade
A Customer
Experience
Real-time backend
operations
Event streaming with Kafka to set Data in Motion:
Continuously processing evolving streams of data in real time with Confluent

13. © 2023, Amazon Web Services, Inc. or its affiliates.
Out-of-box integration with
popular services
Certified and validated by
AWS
AWS Native Services
Top-5 Global ISV for S3 Data Volume
3rd-Party ISV Services
• Amazon RDS
Ready
• AWS Lambda
Ready
• Amazon Redshift
Ready
• AWS PrivateLink
Ready
• AWS Outposts
Ready
Validated Service Designations
Confluent integrations with AWS

14. © 2023, Amazon Web Services, Inc. or its affiliates.
S
Confluent Cloud on AWS – reference architecture
S3

15. © 2023, Amazon Web Services, Inc. or its affiliates.
Confluent + AWS: Accelerate Customer Business Outcomes
Topline Impacting New
Experiences
● Event-driven & real-time
● Unify data across org. w/ Kafka
data fabric (Schema Reg,..)
● AWS Analytics, Redshift, ML
connectors
Mitigate Risk
● Higher Service Quality &
Resilience with 99.99% SLA
● Deep Kafka expertise & innovation
● Elastic billing/pricing
Developer Agility
● Focus on innovation (not data
infrastructure)
● Leverage full Kafka OSS
ecosystem + AWS services
Faster Time to Market
● ~50-75% faster time to market*
● Streamline hybrid cloud
migration with no complex lift-n-
shift
● Maintain business continuity
Lower Kafka TCO
● ~25-50% lower TCO *
● GBps-scale & fast deployments
for global expansion
● Deploy Kafka at scale in 1 week
Maximize ROI
● ~200% ROI per Forrester study
● Save 10s of $Ms with legacy
offload to AWS with Confluent
Replicator
* For customers that don’t already have Kafka based system in-market
* TCO assessment to be analyzed for specific customer scenarios

16. © 2023, Amazon Web Services, Inc. or its affiliates.
Accelerate modernization from on-prem to AWS
Redshift Sink
Lambda Sink
AWS Direct
Connect
Replicator
Legacy Applications
Mainframe
Legacy Data Systems
JDBC / CDC
connectors
Connect
Leverage 120+ Confluent pre-built connectors
Modernize
Value added apps, increase agility, reduce TCO
On-prem AWS Cloud
Bridge
Hybrid cloud streaming
Amazon Athena
AWS Glue
Amazon
SageMaker
AWS Lake
Formation
Amazon
DynamoDB
Amazon
Amazon
Aurora
S3 Sink
Data Streams
Apps
ksqlDB

17. © 2023, Amazon Web Services, Inc. or its affiliates.
Increase developer agility & speed of innovation
Serverless integration
Connect for effortless integrations with Lambda & data stores
AWS serverless platform
Free up backend operation & Infras. management
Apps
Microservices
ksqlDB
Schema
Registry
COMPUTE
AWS
Lambda
Data stores
REST Proxy
& Clients
Source
Connectors
Lambda
Sink
DATA STORES
Amazon
DynamoDB
Amazon
Aurora
STORAGE
Amazon
S3
S3 Sink
ANALYTICS
Amazon
Athena
Amazon
Redshift

18. © 2023, Amazon Web Services, Inc. or its affiliates.
Amazon Redshift Warehousing with Confluent Cloud
Serverless with AWS and Confluent Cloud
Real-time Sentiment Analysis with Confluent
Amazon ElastiCache and Confluent Cloud
confluent.awsworkshop.io
Try it out yourself

19. © 2023, Amazon Web Services, Inc. or its affiliates.
Learn more
Working with streaming data on AWS
https://aws.amazon.com/streaming-data/
Modern Data Architecture on AWS
https://go.aws/3OJDhFk
Build Modern Data Streaming Analytics
Architectures on AWS
https://go.aws/3bt0HAm
Derive Insights from Modern Data
https://go.aws/3xVU3dn

20. © 2023, Amazon Web Services, Inc. or its affiliates.
Free trial available through the
AWS Marketplace
$400 in free credits to spend
during your first 30 days

21. © 2023, Amazon Web Services, Inc. or its affiliates.
© 2023, Amazon Web Services, Inc. or its affiliates.
Thank you!
Nae Ittaboon
ittaboon@amazon.co.th