Will LaForest is the Public Sector CTO for Confluent. In his current position, Will evangelizes the benefits of Apache Kafka, event-driven data in motion architecture, and open-source software is addressing mission challenges in the Government. He has spent 25 years wrangling data at massive scale. His technical career spans diverse areas from software engineering, NoSQL, data science, cloud computing, machine learning, and building statistical visualization software but began with code slinging at DARPA as a teenager. Will holds degrees in mathematics and physics from the University of Virginia. To watch all of the recordings hosted during Scylla Summit 2022 visit our website here: https://www.scylladb.com/summit.

1. An Odyssey to ScyllaDB
and Apache Kafka
Will LaForest
CTO, Americas & Public Sector

2. Will LaForest
■ Confluent
■ MongoDB, MarkLogic
■ Red Hat
■ Independent Advisor on Data Systems
■ Data management, analysis, science
CTO, Americas & Public Sector

3. Data at rest
Batch processing
Queries Against
Accumulated State
At a Point In Time
Databases
Age Old Way of Handling Data

4. Data: The New Currency
of Business Success

5. Shift in Thinking: Event Streams
Diagram or photo
Rich User
experiences
Real-time
events
Real-time
Event Streams
A Sale A shipment
A Trade
Customer
Actions
Data driven
operation
Real-time
Analytics
Microservices
Stream
Processing

6. Event Oriented

7. Command Oriented

8. Event Oriented - Asynchronous

9. Event Oriented - Decoupled

10. Event Oriented - Decoupled

11. ■ Event production
• Anything is a source
• DB Transactions
• Event streams
■ Events flow between between
■ Data in Motion
■ Data at Rest
Modern Architecture Combines Both

12. The Rise of Event Streaming
2010
created at LinkedIn by
Confluent founders
2014
2020
80%
Fortune 100
Companies
trust and use
Apache Kafka

13. Anatomy of a Kafka Topic
1 2 3 4 5 6 8 9
7
Partition 1
Old New
1 2 3 4 5 6 8
7
Partition 0 10
9 11 12
Partition 2 1 2 3 4 5 6 8
7 10
9 11 12
Writes
1 2 3 4 5 6 8
7 10
9 11 12
Producers
Writes
Consumer A
(offset=4)
Consumer B
(offset=7)
Reads

14. Confluent for Data In Motion
Dynamic Performance & Elasticity
Self-Balancing Clusters | Tiered Storage
Flexible DevOps Automation
Operator | Ansible
GUI-driven Mgmt & Monitoring
Control Center
Efficient
Operations at Scale
Freedom of Choice
Committer-driven Expertise
Event Streaming Database
ksqlDB
Rich Pre-built Ecosystem
Connectors | Hub | Schema Registry
Multi-language Development
Non-Java Clients | REST Proxy
Admin REST APIs
Global Resilience
Multi-Region Clusters | Replicator
Cluster Linking
Data Compatibility
Schema Registry | Schema Validation
Enterprise-grade Security
RBAC | FIPS 140-2 | Audit Logs
ARCHITECT
OPERATOR
DEVELOPER
Unrestricted
Developer Productivity
Production-stage
Prerequisites
Fully Managed Cloud Service
Self-managed Software
Training Partners
Enterprise
Support
Professional
Services
Apache Kafka

15. Connecting the Two
Various methods
■ Event produced into Kafka
• Kafka connect sink or data store specific support
■ Transaction against data store
• Hopefully CDC
• Kafka Connectors
■ Dual emission
• Event produces data store transaction AND is published to Kafka

16. ■ ScyllaDB is ultra performant and scalable
• It can keep up with Kafka without over resourcing
• Low latency is important for event driven real-time applications
■ Understands event driven approaches
• Actually investing in open CDC connector for Kafka
• Worked well with them in join customers
Why ScyllaDB Works Well

17. ■ Enable rapid response and
action
■ Become data driven
■ Leverage a wide variety of
different data sets
■ Scale to meet current and
future demands
■ Cloud and on-prem
Government All Domain Analyst System

18. ■ Efforts started with
Neo4J and RDBMS
■ They investigated Atlas
for data lineage
■ Settled on Janus
• Much better results
with Scylla than
Cassandra
Graph Approach

19. Architecture

20. Thank you!
Stay in touch
Will LaForest
@WLaForest
will@confluent.io
developer.confluent.io