In this webinar, you’ll also be introduced to DataStax Apache Kafka Connector, and get a brief demonstration of this groundbreaking technology. You’ll directly experience how this tool can help you stream data from Kafka topics into DataStax Enterprise versions of Cassandra. The future of your organization won’t wait. Register now to reserve your spot in this exciting new webinar. Youtube: https://youtu.be/HmkNb8twUNk

1. Better Together: Apache
Cassandra and Apache Kafka
1

2. 2 © DataStax, All Rights Reserved.
Confidential
Agenda
1 Apache Cassandra and Apache Kafka
2 Better Together – Common Patterns
3 DataStax Kafka Connector
4 Demonstration
5 Resources

3. Your Presenters
3 © DataStax, All Rights Reserved.

4. 4 © DataStax, All Rights Reserved.
Confidential
Agenda
1 Apache Cassandra and Apache Kafka
2 Better Together – Common Patterns
3 DataStax Kafka Connector
4 Demonstration
5 Resources

5. Apache Cassandra Overview
• First developed by Facebook
• Top-level Apache project since 2010
• Partitioned row store
• Distributed, decentralized
• Elastic scalability / high performance
• High availability / fault tolerant
• Tuneable consistency
• Cassandra Query Language (CQL)
© DataStax, All Rights Reserved.5 Apache Cassandra ® Apache Software Foundation

6. Apache Kafka Overview
• First developed by LinkedIn
• Top-level Apache Project since 2012
• Distributed streaming platform
• Used for real-time data pipelines and
streaming applications
• Horizontal scalability / high performance
• High availability / Fault tolerance
• Stream persistence and querying
(KSQL)
• Connect framework
6 © DataStax, All Rights Reserved. Apache Kafka ® Apache Software Foundation

7. Kafka Concepts
• Topics
– Collection of key/value pairs
– Append-only
– Can be partitioned
• Producers
• Consumers
– Separate offsets
7 © DataStax, All Rights Reserved.

8. Kafka Concepts
• Streams applications
– Combined Producer/Consumer
• KSQL
– Query language used by stream
applications
8 © DataStax, All Rights Reserved.

9. Kafka Concepts
• Brokers
• Clusters
• Connect Framework
– Sources
– Sinks
9 © DataStax, All Rights Reserved.

10. Cassandra + Kafka – Similarities and Distinctives
• Concepts in common
– Distributed Systems
– Partitioning / Hashing
– Replication
• Slight differences in implementation
– Multi-DC
– Log-structured
– TTL / retention
• Cassandra excels at…
– High volume, write intensive data storage
workloads at scale
– Suitable as a system of record
– High performance searching via DSE
• Kafka excels at…
– Streaming data to/from services and legacy
data sources
– Acting upon changes in data from multiple
sources (aka pipelines)
10 © DataStax, All Rights Reserved.

11. +
Better Together – using the best of both

12. 12 © DataStax, All Rights Reserved.
Confidential
Agenda
1 Apache Cassandra and Apache Kafka
2 Better Together – Common Patterns
3 DataStax Kafka Connector
4 Demonstration
5 Resources

13. Pattern 1: Cassandra + Kafka in Microservices
13 © DataStax, All Rights Reserved.
Some
Producer
My
microservice
DataStax Enterprise
• Consume
topic(s)
Other
consumers
• Read /
write data
• Publish to
topic(s)

14. KillrVideo Services Suggested
Videos
Service
DataStax Enterprise
DSE Graph
• UserCreated
• YouTubeVideoAdded
• UserRatedVideo • Populate graph
• Graph recommender
traversal
• Read and
write data
User Management, Video
Catalog, Ratings
Cassandra + Kafka – KillrVideo Example

15. Confidential© DataStax, All Rights Reserved.
Pattern 2: Kafka into Cassandra
15

16. 16 © DataStax, All Rights Reserved.
Confidential
Agenda
1 Apache Cassandra and Apache Kafka
2 Better Together – Common Patterns
3 DataStax Kafka Connector
4 Demonstration
5 Resources

17. Why a Kafka Connector ?
Account &
Product Usage
Geolocation
ClickStream
Mobile Device
System of records
Event Sources
…
Topic: stocks-ticks

18. Why a Kafka Connector?
• Spark Streaming = PULL
➢ Enable advanced transformations and
computations
➢ Mode Pull with a dedicated runtime (poll)
18
• Kakfa Connector = PUSH
➢ No extra runtime

19. What is the Kafka Connector ?
Account &
Product Usage
Geolocation
ClickStream
Mobile Device
System of records
Event Sources
…
Topic: stocks-ticks
Kafka Connect
Sources Sinks
HERE

20. What is the Kafka Connector?
• Automatically ingest from Kafka to DSE
– Simple, Fast, Flexible, Secure
• Deployed in the Kafka Connect framework
– Managed through the built-in REST API
• Visibility into running connectors and tasks
• Endpoints for operator tasks
– Automatic rebalancing
• Useful for availability and scaling
20
Kafka Connect
Sources Sinks
HERE

21. What is the Kafka Connector?
• Built by DataStax drivers team
– Best practices for writing to DSE
– Resiliency of DS drivers
21
Kafka Connect
Sources Sinks
HERE

22. What is the Kafka Connector?
22
Kafka Connect
Data Sinks
DataStax Connector
workers
<standalone-worker>.properties
<distributed-worker>.properties
or
reads
Connectors list
<connector>.properties
or
<connector>.json
read
Config (mapping)
instantiate
start

23. 23 © DataStax, All Rights Reserved.
Confidential
Agenda
1 Apache Cassandra and Apache Kafka
2 Better Together – Common Patterns
3 DataStax Kafka Connector
4 Demonstration
5 Resources

24. Academy.datastax.com/downloads
24

25. What versions does this work with?
25
DSE 5.0+
Supported Versions
- DSE 5.0+
- Confluent 3.2+
- Apache Kafka 0.10.2+
Supported offerings
- DS Enterprise
- DS Basic
- DDAC
Confluent Apache Kafka
3.2.x+ 0.10.2.x+
3.3.x+ 0.11.0.x+
4.0.x+ 1.0.x+
4.1.x+ 1.1.x+
5.0.x+ 2.0.x+

26. Docker and Datastax
26 Confidential
• WHERE
– https://hub.docker.com/u/datastax/
– https://github.com/datastax/docker-
images/tree/master/datastax-docker-image-
examples
• We provide
– Dockers images for DSE, studio, Opscenter
– Docker-compose configuration files
– Sample Deployments
• We support
– Installation on dev before 6.7
– Installation on prod from 6.7 (December 2018)

27. https://github.com/clun/kafka-dse/tree/driver2
27 © DataStax, All Rights Reserved. Confidential

28. Demonstration Overview
© DataStax, All Rights Reserved.
Kafka-dse-producer
TickGenerator
StockTickProducer
timer
Alpha Vantage
2181
9092
KAFKA-CONNECT
8083
Sources Sinks
DSE
Kafka-dse-webui
Java-dse-driver
Spring Web Flux

29. 29 © DataStax, All Rights Reserved.
Confidential
Agenda
1 Apache Cassandra and Apache Kafka
2 Better Together – Common Patterns
3 DataStax Kafka Connector
4 Demonstration
5 Resources

30. Documentation and Examples
• Blog
– https://www.datastax.com/2018/12/introducing-the-datastax-apache-kafka-connector
• Download
– https://academy.datastax.com/downloads#connectors
• Docs
– https://docs.datastax.com/en/kafka/doc/index.html
• Demonstration
– https://github.com/clun/kafka-dse/tree/driver2
• Examples
– https://github.com/datastax/kafka-examples
30 © DataStax, All Rights Reserved.

31. DataStax Academy
• Free self-paced courses
• DS201: Apache Cassandra™
• DS210: Operations
• DS220: Data Modeling
• DS310: Search
• DS320: Analytics
• DS330: Graph
• Kafka Connector Getting Started
31 © DataStax, All Rights Reserved.
https://academy.datastax.com

32. Live Coding on Twitch
• Live coding sessions with advocates and
guests
• Working through the challenges of
building distributed systems
• Join the conversation and ask questions
• Twitch Rewind: Kafka Connector
– https://www.youtube.com/watch?v=2_BidD
K5zGE
https://www.twitch.tv/datastaxacademy
32 © DataStax, All Rights Reserved.

33. Join us at Accelerate!
33 © DataStax, All Rights Reserved.
http://www.datastax.com/accelerate
Discount Code: ADVOCATE20

34. Cassandra + Kafka at Accelerate!
34 © DataStax, All Rights Reserved.
…and additional sessions

35. Thank you
35 © DataStax, All Rights Reserved.
Confidential