SlideShare a Scribd company logo
1 of 54
Download to read offline
©2013 DataStax Confidential. Do not distribute without consent.
@PatrickMcFadin
Patrick McFadin

Chief Evangelist, DataStax
Advanced Cassandra
1
Does Apache Cassandra Work?
Motivations
Cassandra is not…
6
A Data Ocean or Pond., Lake
An In-Memory Database
A Key-Value Store
A magical database unicorn that farts rainbows
7
When to use…
Loose data model (joins, sub-selects)
Absolute consistency (aka gotta have ACID)
No need to use anything else
You’ll miss the long, candle lit dinners with your Oracle rep
that always end with “what’s your budget look like this
year?”
Oracle, MySQL, Postgres or <RDBMS>
Uptime is a top priority
Unpredictable or high scaling requirements
Workload is transactional
Willing to put the time or effort into understanding how Cassandra works
and how to use it.
8
When to use…
Use Oracle when you want to count your money.
Use Cassandra when you want to make money.
Cassandra
Copy n Paste your relational model
APACHE
CASSANDRA
1000 Node Cluster
Scaling up
Stick the landing
12
Going to deploy in production!
Not sure about this!
Done!
Topology considerations
Replication Strategy
CREATE KEYSPACE killrvideo WITH
REPLICATION = { 'class' : 'SimpleStrategy', 'replication_factor' : 3 };
Strategy
Copies
Topology considerations
• Default
• One data center
SimpleStrategy
NetworkTopologyStrategy
• Use for multi-data center
• Just use this always
NetworkTopologyStrategy
CREATE KEYSPACE Product_Catalog WITH
REPLICATION = { 'class' : 'NetworkTopologyStrategy', 'replication_factor' : 3 };
CREATE KEYSPACE EU_Customer_Data WITH
REPLICATION = { 'class' : 'NetworkTopologyStrategy',
'eu1' : 3
‘eu2’ : 3
‘us1’ : 0 };
Symmetric
Asymmetric
No copies in the US
Application
• Closer to customers
• No downtime
Product_Catalog RF=3
Product_Catalog RF=3 EU_Customer_Data RF=3
EU_Customer_Data RF=0
Product_Catalog RF=3
EU_Customer_Data RF=3
Snitches
Snitches
DC1
DC1: RF=3
Node Primary Replica Replica
10.0.0.1 00-25 76-100 51-75
10.0.0.2 26-50 00-25 76-100
10.0.0.3 51-75 26-50 00-25
10.0.0.4 76-100 51-75 26-50
10.0.0.1
00-25
10.0.0.4
76-100
10.0.0.2
26-50
10.0.0.3
51-75
76-100
51-75
00-25
76-100
26-50
00-25
51-75
26-50
Client
Where do I place
this data?
?
Dynamic Snitching
Route based on node performance
Snitches
SimpleSnitch
GossipingPropertyFileSnitch
RackInferringSnitch
PropertyFileSnitch
EC2Snitch
GoogleCloudSnitch
CloudStackSnitch
EC2MultiRegionSnitch
Snitches
• Most typically used in production
• Absolute placement
GossipingPropertyFileSnitch
cassandra-rackdc.properties
dc=DC1
rack=RAC1
Booting a datacenter
DC1
DC1: RF=3
Node Primary Replica Replica
10.0.0.1 00-25 76-100 51-75
10.0.0.2 26-50 00-25 76-100
10.0.0.3 51-75 26-50 00-25
10.0.0.4 76-100 51-75 26-50
10.0.0.1
00-25
10.0.0.4
76-100
10.0.0.2
26-50
10.0.0.3
51-75
76-100
51-75
00-25
76-100
26-50
00-25
51-75
26-50
DC2
Pre-check
• Use NetworkTopologyStrategy
• In cassandra.yaml
• auto_bootstrap: false
• add seeds from other DC
• Set node location for Snitch
• GossipingPropertyFileSnitch:
cassandra-rackdc.properties
• PropertyFileSnitch: cassandra-
topology.properties
Booting a datacenter
DC1
DC1: RF=3
Node Primary Replica Replica
10.0.0.1 00-25 76-100 51-75
10.0.0.2 26-50 00-25 76-100
10.0.0.3 51-75 26-50 00-25
10.0.0.4 76-100 51-75 26-50
10.0.0.1
00-25
10.0.0.4
76-100
10.0.0.2
26-50
10.0.0.3
51-75
76-100
51-75
00-25
76-100
26-50
00-25
51-75
26-50
DC2
10.1.0.1
00-25
10.1.0.4
76-100
10.1.0.2
26-50
10.1.0.3
51-75
76-100
51-75
00-25
76-100
26-50
00-25
51-75
26-50
Node Primary Replica Replica
10.0.0.1 00-25 76-100 51-75
10.0.0.2 26-50 00-25 76-100
10.0.0.3 51-75 26-50 00-25
10.0.0.4 76-100 51-75 26-50
DC2: RF=3
ALTER KEYSPACE
Booting a datacenter
DC1
DC1: RF=3
Node Primary Replica Replica
10.0.0.1 00-25 76-100 51-75
10.0.0.2 26-50 00-25 76-100
10.0.0.3 51-75 26-50 00-25
10.0.0.4 76-100 51-75 26-50
10.0.0.1
00-25
10.0.0.4
76-100
10.0.0.2
26-50
10.0.0.3
51-75
76-100
51-75
00-25
76-100
26-50
00-25
51-75
26-50
DC2
10.1.0.1
00-25
10.1.0.4
76-100
10.1.0.2
26-50
10.1.0.3
51-75
76-100
51-75
00-25
76-100
26-50
00-25
51-75
26-50
Node Primary Replica Replica
10.0.0.1 00-25 76-100 51-75
10.0.0.2 26-50 00-25 76-100
10.0.0.3 51-75 26-50 00-25
10.0.0.4 76-100 51-75 26-50
DC2: RF=3
nodetool rebuild
Security
NoSQL == No Security
User Auth
Step 1 Turn it on
cassandra.yaml
authorizer:PasswordAuthorizerAllowAllAuthorizer
authenticator:AllowAllAuthenticatorPasswordAuthenticator
User Auth
cqlsh -u cassandra -p cassandra
Step 2 Create users
cqlsh> create user dude with password 'manager' superuser;
cqlsh> create user worker with password 'newhire';
cqlsh> list users;
name | super
----------+-------
cassandra | True
worker | False
dude | True
User Auth
cqlsh -u cassandra -p cassandra
Step 3 Grant permissions
cqlsh> create user ro_user with password '1234567';
cqlsh> grant all on killrvideo.user to dude;
cqlsh> grant select on killrvideo.user to ro_user;
SSL
http://thelastpickle.com/blog/2015/09/30/hardening-cassandra-step-by-step-part-1-server-to-server.html
10.0.0.1
10.0.0.4 10.0.0.2
10.0.0.3
• Create SSL certificates
• Copy to each server
• Start each node
Prepared Statements
• Built for speed an efficiency
How they work: Prepare
SELECT * FROM user WHERE id = ?
10.0.0.1
00-25
10.0.0.4
76-100
10.0.0.2
26-50
10.0.0.3
51-75
Client
Prepare
Parsed
Hashed Cached
Prepared Statement
How they work: Bind
id = 1 + PreparedStatement Hash
10.0.0.1
00-25
10.0.0.4
76-100
10.0.0.2
26-50
10.0.0.3
51-75
Client
Bind & Execute
Combine
Pre-parsed Query and
Variable
Execute
Result?
How to Prepare(Statements)
PreparedStatement userSelect = session.prepare(“SELECT * FROM user WHERE id = ?”);
BoundStatement userSelectStatement = new BoundStatement(userSelect);
session.execute(userSelectStatement.bind(1));
prepared_stmt = session.prepare (“SELECT * FROM user WHERE id = ?”)
bound_stmt = prepared_stmt.bind([1])
session.execute(bound_stmt)
Java
Python
Don’t do this
for (int i = 1; i < 100; i++) {
PreparedStatement userSelect = session.prepare(“SELECT * FROM user WHERE id = ?”);
BoundStatement userSelectStatement = new BoundStatement(userSelect);
session.execute(userSelectStatement.bind(1));
}
Execute vs Execute Async
• Very subtle difference
• Blocking vs non-blocking call
VS
Async
• Request pipelining
• One connection for requests
• Responses return whenever
Async
for (…) {
future = executeAsync(statement)
}
10.0.0.1
00-25
10.0.0.4
76-100
10.0.0.2
26-50
10.0.0.3
51-75
Client
Do something
for (…) {
result = future.get
}
Block
Batch vs Execute Async
VS
(Potentially)
Load Balancing Policies
cluster = Cluster
.builder()
.addContactPoint("192.168.0.30")
.withQueryOptions(new QueryOptions().setConsistencyLevel(ConsistencyLevel.ONE)
.withRetryPolicy(DefaultRetryPolicy.INSTANCE)
.withLoadBalancingPolicy(new TokenAwarePolicy(new DCAwareRoundRobinPolicy()))
.build();
session = cluster.connect("demo");
Data Locality
DC1
DC1: RF=3
Node Primary Replica Replica
10.0.0.1 00-25 76-100 51-75
10.0.0.2 26-50 00-25 76-100
10.0.0.3 51-75 26-50 00-25
10.0.0.4 76-100 51-75 26-50
10.0.0.1
00-25
10.0.0.4
76-100
10.0.0.2
26-50
10.0.0.3
51-75
76-100
51-75
00-25
76-100
26-50
00-25
51-75
26-50
Client
Read partition
15
DC2
10.1.0.1
00-25
10.1.0.4
76-100
10.1.0.2
26-50
10.1.0.3
51-75
76-100
51-75
00-25
76-100
26-50
00-25
51-75
26-50
Node Primary Replica Replica
10.0.0.1 00-25 76-100 51-75
10.0.0.2 26-50 00-25 76-100
10.0.0.3 51-75 26-50 00-25
10.0.0.4 76-100 51-75 26-50
DC2: RF=3
Client
Read partition
15
Batch (Logged)
• All statements collected on client
• Sent in one shot
• All done on 1 node
Batch is accepted
All actions are logged on
two replicas
Statements executed in
sequence
Results are collected and
returned
Batches: The good
• Great for denormalized inserts/updates
// Looking from the video side to many users
CREATE TABLE comments_by_video (
videoid uuid,
commentid timeuuid,
userid uuid,
comment text,
PRIMARY KEY (videoid, commentid)
) WITH CLUSTERING ORDER BY (commentid DESC);
// looking from the user side to many videos
CREATE TABLE comments_by_user (
userid uuid,
commentid timeuuid,
videoid uuid,
comment text,
PRIMARY KEY (userid, commentid)
) WITH CLUSTERING ORDER BY (commentid DESC);
Batches: The good
• Both inserts are run
• On failure, the batch log will replay
BEGIN BATCH
INSERT INTO comments_by_video (videoid, userid, commentid, comment)
VALUES (99051fe9-6a9c-46c2-b949-38ef78858dd0,d0f60aa8-54a9-4840-b70c-fe562b68842b,now(), 'Worst. Video. Ever.')
INSERT INTO comments_by_video (videoid, userid, commentid, comment)
VALUES (99051fe9-6a9c-46c2-b949-38ef78858dd0,d0f60aa8-54a9-4840-b70c-fe562b68842b,now(), 'Worst. Video. Ever.')
APPLY BATCH;
Batches: The bad
“I was doing a load test and nodes started blinking offline”
“Were you using a batch by any chance?”
“Why yes I was! How did you know?”
“How big was each batch?”
“1000 inserts each”
Batches: The bad
BEGIN BATCH
1000 inserts
APPLY BATCH;
10.0.0.1
00-25
10.0.0.4
76-100
10.0.0.2
26-50
10.0.0.3
51-75
Client
Batches: The rules
• Keep them small and for atomicity
CASSANDRA-6487 - Warn on large batches (5Kb default)
CASSANDRA-8011 - Fail on large batches (50Kb default)
The alternative
BEGIN BATCH
1000 inserts
APPLY BATCH;
while() {
future = session.executeAsync(statement)
}
Instead of:
Do this:
Old Row cache: The problem
• Reads an entire storage row of data
ID = 1
Partition Key
(Storage Row Key)
2014-09-08 12:00:00 :
name
SFO
2014-09-08 12:00:00 :
temp
63.4
2014-09-08 12:01:00 :
name
SFO
2014-09-08 12:00:00 :
temp
63.9
2014-09-08 12:02:00 :
name
SFO
2014-09-08 12:00:00 :
temp
64.0
Need this
Caches this
New Row Cache: The solution
• Stores just a few CQL rows
ID = 1
Partition Key
(Storage Row Key)
2014-09-08 12:00:00 :
name
SFO
2014-09-08 12:00:00 :
temp
63.4
2014-09-08 12:01:00 :
name
SFO
2014-09-08 12:00:00 :
temp
63.9
2014-09-08 12:02:00 :
name
SFO
2014-09-08 12:00:00 :
temp
64.0
Need this
Caches this
Using row cache
CREATE TABLE user_search_history_with_cache (
id int,
search_time timestamp,
search_text text,
search_results int,
PRIMARY KEY (id, search_time)
) WITH CLUSTERING ORDER BY (search_time DESC)
AND caching = { 'keys' : 'ALL', 'rows_per_partition' : '20' };
Perf increase
95th ms
Requests
Go make something awesome
Thank you!
Bring the questions
Follow me on twitter
@PatrickMcFadin

More Related Content

What's hot

DataStax | Effective Testing in DSE (Lessons Learned) (Predrag Knezevic) | Ca...
DataStax | Effective Testing in DSE (Lessons Learned) (Predrag Knezevic) | Ca...DataStax | Effective Testing in DSE (Lessons Learned) (Predrag Knezevic) | Ca...
DataStax | Effective Testing in DSE (Lessons Learned) (Predrag Knezevic) | Ca...
DataStax
 
Enabling Search in your Cassandra Application with DataStax Enterprise
Enabling Search in your Cassandra Application with DataStax EnterpriseEnabling Search in your Cassandra Application with DataStax Enterprise
Enabling Search in your Cassandra Application with DataStax Enterprise
DataStax Academy
 
What We Learned About Cassandra While Building go90 (Christopher Webster & Th...
What We Learned About Cassandra While Building go90 (Christopher Webster & Th...What We Learned About Cassandra While Building go90 (Christopher Webster & Th...
What We Learned About Cassandra While Building go90 (Christopher Webster & Th...
DataStax
 
The Promise and Perils of Encrypting Cassandra Data (Ameesh Divatia, Baffle, ...
The Promise and Perils of Encrypting Cassandra Data (Ameesh Divatia, Baffle, ...The Promise and Perils of Encrypting Cassandra Data (Ameesh Divatia, Baffle, ...
The Promise and Perils of Encrypting Cassandra Data (Ameesh Divatia, Baffle, ...
DataStax
 
Introduction to Data Modeling with Apache Cassandra
Introduction to Data Modeling with Apache CassandraIntroduction to Data Modeling with Apache Cassandra
Introduction to Data Modeling with Apache Cassandra
DataStax Academy
 

What's hot (20)

Introduction to Cassandra Architecture
Introduction to Cassandra ArchitectureIntroduction to Cassandra Architecture
Introduction to Cassandra Architecture
 
Coursera Cassandra Driver
Coursera Cassandra DriverCoursera Cassandra Driver
Coursera Cassandra Driver
 
DataStax | Effective Testing in DSE (Lessons Learned) (Predrag Knezevic) | Ca...
DataStax | Effective Testing in DSE (Lessons Learned) (Predrag Knezevic) | Ca...DataStax | Effective Testing in DSE (Lessons Learned) (Predrag Knezevic) | Ca...
DataStax | Effective Testing in DSE (Lessons Learned) (Predrag Knezevic) | Ca...
 
Software Development with Apache Cassandra
Software Development with Apache CassandraSoftware Development with Apache Cassandra
Software Development with Apache Cassandra
 
Introduction to data modeling with apache cassandra
Introduction to data modeling with apache cassandraIntroduction to data modeling with apache cassandra
Introduction to data modeling with apache cassandra
 
Cassandra 3.0 Data Modeling
Cassandra 3.0 Data ModelingCassandra 3.0 Data Modeling
Cassandra 3.0 Data Modeling
 
Enabling Search in your Cassandra Application with DataStax Enterprise
Enabling Search in your Cassandra Application with DataStax EnterpriseEnabling Search in your Cassandra Application with DataStax Enterprise
Enabling Search in your Cassandra Application with DataStax Enterprise
 
C* Summit 2013: Can't we all just get along? MariaDB and Cassandra by Colin C...
C* Summit 2013: Can't we all just get along? MariaDB and Cassandra by Colin C...C* Summit 2013: Can't we all just get along? MariaDB and Cassandra by Colin C...
C* Summit 2013: Can't we all just get along? MariaDB and Cassandra by Colin C...
 
How to Bulletproof Your Scylla Deployment
How to Bulletproof Your Scylla DeploymentHow to Bulletproof Your Scylla Deployment
How to Bulletproof Your Scylla Deployment
 
Cassandra Basics, Counters and Time Series Modeling
Cassandra Basics, Counters and Time Series ModelingCassandra Basics, Counters and Time Series Modeling
Cassandra Basics, Counters and Time Series Modeling
 
Advanced Operations
Advanced OperationsAdvanced Operations
Advanced Operations
 
What We Learned About Cassandra While Building go90 (Christopher Webster & Th...
What We Learned About Cassandra While Building go90 (Christopher Webster & Th...What We Learned About Cassandra While Building go90 (Christopher Webster & Th...
What We Learned About Cassandra While Building go90 (Christopher Webster & Th...
 
Time series with apache cassandra strata
Time series with apache cassandra   strataTime series with apache cassandra   strata
Time series with apache cassandra strata
 
Advanced data modeling with apache cassandra
Advanced data modeling with apache cassandraAdvanced data modeling with apache cassandra
Advanced data modeling with apache cassandra
 
Webinar: Getting Started with Apache Cassandra
Webinar: Getting Started with Apache CassandraWebinar: Getting Started with Apache Cassandra
Webinar: Getting Started with Apache Cassandra
 
The Promise and Perils of Encrypting Cassandra Data (Ameesh Divatia, Baffle, ...
The Promise and Perils of Encrypting Cassandra Data (Ameesh Divatia, Baffle, ...The Promise and Perils of Encrypting Cassandra Data (Ameesh Divatia, Baffle, ...
The Promise and Perils of Encrypting Cassandra Data (Ameesh Divatia, Baffle, ...
 
Troubleshooting Cassandra (J.B. Langston, DataStax) | C* Summit 2016
Troubleshooting Cassandra (J.B. Langston, DataStax) | C* Summit 2016Troubleshooting Cassandra (J.B. Langston, DataStax) | C* Summit 2016
Troubleshooting Cassandra (J.B. Langston, DataStax) | C* Summit 2016
 
Introduction to Data Modeling with Apache Cassandra
Introduction to Data Modeling with Apache CassandraIntroduction to Data Modeling with Apache Cassandra
Introduction to Data Modeling with Apache Cassandra
 
Bulk Loading into Cassandra
Bulk Loading into CassandraBulk Loading into Cassandra
Bulk Loading into Cassandra
 
C* Summit 2013: The World's Next Top Data Model by Patrick McFadin
C* Summit 2013: The World's Next Top Data Model by Patrick McFadinC* Summit 2013: The World's Next Top Data Model by Patrick McFadin
C* Summit 2013: The World's Next Top Data Model by Patrick McFadin
 

Similar to Advanced Cassandra

Dr Elephant: LinkedIn's Self-Service System for Detecting and Treating Hadoop...
Dr Elephant: LinkedIn's Self-Service System for Detecting and Treating Hadoop...Dr Elephant: LinkedIn's Self-Service System for Detecting and Treating Hadoop...
Dr Elephant: LinkedIn's Self-Service System for Detecting and Treating Hadoop...
DataWorks Summit
 
Trouble shooting apachecloudstack
Trouble shooting apachecloudstackTrouble shooting apachecloudstack
Trouble shooting apachecloudstack
Sailaja Sunil
 

Similar to Advanced Cassandra (20)

CloudStack - Top 5 Technical Issues and Troubleshooting
CloudStack - Top 5 Technical Issues and TroubleshootingCloudStack - Top 5 Technical Issues and Troubleshooting
CloudStack - Top 5 Technical Issues and Troubleshooting
 
Copy Data Management for the DBA
Copy Data Management for the DBACopy Data Management for the DBA
Copy Data Management for the DBA
 
Denver SQL Saturday The Next Frontier
Denver SQL Saturday The Next FrontierDenver SQL Saturday The Next Frontier
Denver SQL Saturday The Next Frontier
 
Scaling asp.net websites to millions of users
Scaling asp.net websites to millions of usersScaling asp.net websites to millions of users
Scaling asp.net websites to millions of users
 
Confoo 2021 -- MySQL New Features
Confoo 2021 -- MySQL New FeaturesConfoo 2021 -- MySQL New Features
Confoo 2021 -- MySQL New Features
 
6 tips for improving ruby performance
6 tips for improving ruby performance6 tips for improving ruby performance
6 tips for improving ruby performance
 
Dr Elephant: LinkedIn's Self-Service System for Detecting and Treating Hadoop...
Dr Elephant: LinkedIn's Self-Service System for Detecting and Treating Hadoop...Dr Elephant: LinkedIn's Self-Service System for Detecting and Treating Hadoop...
Dr Elephant: LinkedIn's Self-Service System for Detecting and Treating Hadoop...
 
Oracle to Cassandra Core Concepts Guide Pt. 2
Oracle to Cassandra Core Concepts Guide Pt. 2Oracle to Cassandra Core Concepts Guide Pt. 2
Oracle to Cassandra Core Concepts Guide Pt. 2
 
Instaclustr webinar 2017 feb 08 japan
Instaclustr webinar 2017 feb 08   japanInstaclustr webinar 2017 feb 08   japan
Instaclustr webinar 2017 feb 08 japan
 
JAX London 2021: Jumpstart Your Cloud Native Development: An Overview of Prac...
JAX London 2021: Jumpstart Your Cloud Native Development: An Overview of Prac...JAX London 2021: Jumpstart Your Cloud Native Development: An Overview of Prac...
JAX London 2021: Jumpstart Your Cloud Native Development: An Overview of Prac...
 
Azure from scratch part 4
Azure from scratch part 4Azure from scratch part 4
Azure from scratch part 4
 
Whatever it takes - Fixing SQLIA and XSS in the process
Whatever it takes - Fixing SQLIA and XSS in the processWhatever it takes - Fixing SQLIA and XSS in the process
Whatever it takes - Fixing SQLIA and XSS in the process
 
AppFabric Velocity
AppFabric VelocityAppFabric Velocity
AppFabric Velocity
 
Processing 50,000 Events Per Second with Cassandra and Spark (Ben Slater, Ins...
Processing 50,000 Events Per Second with Cassandra and Spark (Ben Slater, Ins...Processing 50,000 Events Per Second with Cassandra and Spark (Ben Slater, Ins...
Processing 50,000 Events Per Second with Cassandra and Spark (Ben Slater, Ins...
 
Processing 50,000 events per second with Cassandra and Spark
Processing 50,000 events per second with Cassandra and SparkProcessing 50,000 events per second with Cassandra and Spark
Processing 50,000 events per second with Cassandra and Spark
 
Processing 50,000 events per second with Cassandra and Spark
Processing 50,000 events per second with Cassandra and SparkProcessing 50,000 events per second with Cassandra and Spark
Processing 50,000 events per second with Cassandra and Spark
 
VMworld 2013: Part 2: How to Build a Self-Healing Data Center with vCenter Or...
VMworld 2013: Part 2: How to Build a Self-Healing Data Center with vCenter Or...VMworld 2013: Part 2: How to Build a Self-Healing Data Center with vCenter Or...
VMworld 2013: Part 2: How to Build a Self-Healing Data Center with vCenter Or...
 
Trouble shooting apachecloudstack
Trouble shooting apachecloudstackTrouble shooting apachecloudstack
Trouble shooting apachecloudstack
 
Real World Lessons on the Pain Points of Node.JS Application
Real World Lessons on the Pain Points of Node.JS ApplicationReal World Lessons on the Pain Points of Node.JS Application
Real World Lessons on the Pain Points of Node.JS Application
 
NYC Cassandra Day - Java Intro
NYC Cassandra Day - Java IntroNYC Cassandra Day - Java Intro
NYC Cassandra Day - Java Intro
 

More from DataStax Academy

Cassandra on Docker @ Walmart Labs
Cassandra on Docker @ Walmart LabsCassandra on Docker @ Walmart Labs
Cassandra on Docker @ Walmart Labs
DataStax Academy
 
Cassandra Adoption on Cisco UCS & Open stack
Cassandra Adoption on Cisco UCS & Open stackCassandra Adoption on Cisco UCS & Open stack
Cassandra Adoption on Cisco UCS & Open stack
DataStax Academy
 
Cassandra @ Netflix: Monitoring C* at Scale, Gossip and Tickler & Python
Cassandra @ Netflix: Monitoring C* at Scale, Gossip and Tickler & PythonCassandra @ Netflix: Monitoring C* at Scale, Gossip and Tickler & Python
Cassandra @ Netflix: Monitoring C* at Scale, Gossip and Tickler & Python
DataStax Academy
 
Standing Up Your First Cluster
Standing Up Your First ClusterStanding Up Your First Cluster
Standing Up Your First Cluster
DataStax Academy
 
Real Time Analytics with Dse
Real Time Analytics with DseReal Time Analytics with Dse
Real Time Analytics with Dse
DataStax Academy
 

More from DataStax Academy (20)

Forrester CXNYC 2017 - Delivering great real-time cx is a true craft
Forrester CXNYC 2017 - Delivering great real-time cx is a true craftForrester CXNYC 2017 - Delivering great real-time cx is a true craft
Forrester CXNYC 2017 - Delivering great real-time cx is a true craft
 
Introduction to DataStax Enterprise Graph Database
Introduction to DataStax Enterprise Graph DatabaseIntroduction to DataStax Enterprise Graph Database
Introduction to DataStax Enterprise Graph Database
 
Introduction to DataStax Enterprise Advanced Replication with Apache Cassandra
Introduction to DataStax Enterprise Advanced Replication with Apache CassandraIntroduction to DataStax Enterprise Advanced Replication with Apache Cassandra
Introduction to DataStax Enterprise Advanced Replication with Apache Cassandra
 
Cassandra on Docker @ Walmart Labs
Cassandra on Docker @ Walmart LabsCassandra on Docker @ Walmart Labs
Cassandra on Docker @ Walmart Labs
 
Cassandra Adoption on Cisco UCS & Open stack
Cassandra Adoption on Cisco UCS & Open stackCassandra Adoption on Cisco UCS & Open stack
Cassandra Adoption on Cisco UCS & Open stack
 
Data Modeling for Apache Cassandra
Data Modeling for Apache CassandraData Modeling for Apache Cassandra
Data Modeling for Apache Cassandra
 
Production Ready Cassandra
Production Ready CassandraProduction Ready Cassandra
Production Ready Cassandra
 
Cassandra @ Netflix: Monitoring C* at Scale, Gossip and Tickler & Python
Cassandra @ Netflix: Monitoring C* at Scale, Gossip and Tickler & PythonCassandra @ Netflix: Monitoring C* at Scale, Gossip and Tickler & Python
Cassandra @ Netflix: Monitoring C* at Scale, Gossip and Tickler & Python
 
Cassandra @ Sony: The good, the bad, and the ugly part 1
Cassandra @ Sony: The good, the bad, and the ugly part 1Cassandra @ Sony: The good, the bad, and the ugly part 1
Cassandra @ Sony: The good, the bad, and the ugly part 1
 
Cassandra @ Sony: The good, the bad, and the ugly part 2
Cassandra @ Sony: The good, the bad, and the ugly part 2Cassandra @ Sony: The good, the bad, and the ugly part 2
Cassandra @ Sony: The good, the bad, and the ugly part 2
 
Standing Up Your First Cluster
Standing Up Your First ClusterStanding Up Your First Cluster
Standing Up Your First Cluster
 
Real Time Analytics with Dse
Real Time Analytics with DseReal Time Analytics with Dse
Real Time Analytics with Dse
 
Cassandra Core Concepts
Cassandra Core ConceptsCassandra Core Concepts
Cassandra Core Concepts
 
Bad Habits Die Hard
Bad Habits Die Hard Bad Habits Die Hard
Bad Habits Die Hard
 
Getting Started with Graph Databases
Getting Started with Graph DatabasesGetting Started with Graph Databases
Getting Started with Graph Databases
 
Cassandra Data Maintenance with Spark
Cassandra Data Maintenance with SparkCassandra Data Maintenance with Spark
Cassandra Data Maintenance with Spark
 
Analytics with Spark and Cassandra
Analytics with Spark and CassandraAnalytics with Spark and Cassandra
Analytics with Spark and Cassandra
 
Make 2016 your year of SMACK talk
Make 2016 your year of SMACK talkMake 2016 your year of SMACK talk
Make 2016 your year of SMACK talk
 
Client Drivers and Cassandra, the Right Way
Client Drivers and Cassandra, the Right WayClient Drivers and Cassandra, the Right Way
Client Drivers and Cassandra, the Right Way
 
Tales From The Front: An Architecture For Multi-Data Center Scalable Applicat...
Tales From The Front: An Architecture For Multi-Data Center Scalable Applicat...Tales From The Front: An Architecture For Multi-Data Center Scalable Applicat...
Tales From The Front: An Architecture For Multi-Data Center Scalable Applicat...
 

Recently uploaded

Finding Java's Hidden Performance Traps @ DevoxxUK 2024
Finding Java's Hidden Performance Traps @ DevoxxUK 2024Finding Java's Hidden Performance Traps @ DevoxxUK 2024
Finding Java's Hidden Performance Traps @ DevoxxUK 2024
Victor Rentea
 
Cloud Frontiers: A Deep Dive into Serverless Spatial Data and FME
Cloud Frontiers:  A Deep Dive into Serverless Spatial Data and FMECloud Frontiers:  A Deep Dive into Serverless Spatial Data and FME
Cloud Frontiers: A Deep Dive into Serverless Spatial Data and FME
Safe Software
 
Why Teams call analytics are critical to your entire business
Why Teams call analytics are critical to your entire businessWhy Teams call analytics are critical to your entire business
Why Teams call analytics are critical to your entire business
panagenda
 
Modular Monolith - a Practical Alternative to Microservices @ Devoxx UK 2024
Modular Monolith - a Practical Alternative to Microservices @ Devoxx UK 2024Modular Monolith - a Practical Alternative to Microservices @ Devoxx UK 2024
Modular Monolith - a Practical Alternative to Microservices @ Devoxx UK 2024
Victor Rentea
 

Recently uploaded (20)

"I see eyes in my soup": How Delivery Hero implemented the safety system for ...
"I see eyes in my soup": How Delivery Hero implemented the safety system for ..."I see eyes in my soup": How Delivery Hero implemented the safety system for ...
"I see eyes in my soup": How Delivery Hero implemented the safety system for ...
 
presentation ICT roal in 21st century education
presentation ICT roal in 21st century educationpresentation ICT roal in 21st century education
presentation ICT roal in 21st century education
 
Apidays New York 2024 - Scaling API-first by Ian Reasor and Radu Cotescu, Adobe
Apidays New York 2024 - Scaling API-first by Ian Reasor and Radu Cotescu, AdobeApidays New York 2024 - Scaling API-first by Ian Reasor and Radu Cotescu, Adobe
Apidays New York 2024 - Scaling API-first by Ian Reasor and Radu Cotescu, Adobe
 
Finding Java's Hidden Performance Traps @ DevoxxUK 2024
Finding Java's Hidden Performance Traps @ DevoxxUK 2024Finding Java's Hidden Performance Traps @ DevoxxUK 2024
Finding Java's Hidden Performance Traps @ DevoxxUK 2024
 
ICT role in 21st century education and its challenges
ICT role in 21st century education and its challengesICT role in 21st century education and its challenges
ICT role in 21st century education and its challenges
 
Cloud Frontiers: A Deep Dive into Serverless Spatial Data and FME
Cloud Frontiers:  A Deep Dive into Serverless Spatial Data and FMECloud Frontiers:  A Deep Dive into Serverless Spatial Data and FME
Cloud Frontiers: A Deep Dive into Serverless Spatial Data and FME
 
EMPOWERMENT TECHNOLOGY GRADE 11 QUARTER 2 REVIEWER
EMPOWERMENT TECHNOLOGY GRADE 11 QUARTER 2 REVIEWEREMPOWERMENT TECHNOLOGY GRADE 11 QUARTER 2 REVIEWER
EMPOWERMENT TECHNOLOGY GRADE 11 QUARTER 2 REVIEWER
 
Understanding the FAA Part 107 License ..
Understanding the FAA Part 107 License ..Understanding the FAA Part 107 License ..
Understanding the FAA Part 107 License ..
 
Spring Boot vs Quarkus the ultimate battle - DevoxxUK
Spring Boot vs Quarkus the ultimate battle - DevoxxUKSpring Boot vs Quarkus the ultimate battle - DevoxxUK
Spring Boot vs Quarkus the ultimate battle - DevoxxUK
 
TrustArc Webinar - Unlock the Power of AI-Driven Data Discovery
TrustArc Webinar - Unlock the Power of AI-Driven Data DiscoveryTrustArc Webinar - Unlock the Power of AI-Driven Data Discovery
TrustArc Webinar - Unlock the Power of AI-Driven Data Discovery
 
Introduction to use of FHIR Documents in ABDM
Introduction to use of FHIR Documents in ABDMIntroduction to use of FHIR Documents in ABDM
Introduction to use of FHIR Documents in ABDM
 
Apidays New York 2024 - Passkeys: Developing APIs to enable passwordless auth...
Apidays New York 2024 - Passkeys: Developing APIs to enable passwordless auth...Apidays New York 2024 - Passkeys: Developing APIs to enable passwordless auth...
Apidays New York 2024 - Passkeys: Developing APIs to enable passwordless auth...
 
Connector Corner: Accelerate revenue generation using UiPath API-centric busi...
Connector Corner: Accelerate revenue generation using UiPath API-centric busi...Connector Corner: Accelerate revenue generation using UiPath API-centric busi...
Connector Corner: Accelerate revenue generation using UiPath API-centric busi...
 
Why Teams call analytics are critical to your entire business
Why Teams call analytics are critical to your entire businessWhy Teams call analytics are critical to your entire business
Why Teams call analytics are critical to your entire business
 
Platformless Horizons for Digital Adaptability
Platformless Horizons for Digital AdaptabilityPlatformless Horizons for Digital Adaptability
Platformless Horizons for Digital Adaptability
 
Strategies for Landing an Oracle DBA Job as a Fresher
Strategies for Landing an Oracle DBA Job as a FresherStrategies for Landing an Oracle DBA Job as a Fresher
Strategies for Landing an Oracle DBA Job as a Fresher
 
Modular Monolith - a Practical Alternative to Microservices @ Devoxx UK 2024
Modular Monolith - a Practical Alternative to Microservices @ Devoxx UK 2024Modular Monolith - a Practical Alternative to Microservices @ Devoxx UK 2024
Modular Monolith - a Practical Alternative to Microservices @ Devoxx UK 2024
 
CNIC Information System with Pakdata Cf In Pakistan
CNIC Information System with Pakdata Cf In PakistanCNIC Information System with Pakdata Cf In Pakistan
CNIC Information System with Pakdata Cf In Pakistan
 
Apidays New York 2024 - The Good, the Bad and the Governed by David O'Neill, ...
Apidays New York 2024 - The Good, the Bad and the Governed by David O'Neill, ...Apidays New York 2024 - The Good, the Bad and the Governed by David O'Neill, ...
Apidays New York 2024 - The Good, the Bad and the Governed by David O'Neill, ...
 
Rising Above_ Dubai Floods and the Fortitude of Dubai International Airport.pdf
Rising Above_ Dubai Floods and the Fortitude of Dubai International Airport.pdfRising Above_ Dubai Floods and the Fortitude of Dubai International Airport.pdf
Rising Above_ Dubai Floods and the Fortitude of Dubai International Airport.pdf
 

Advanced Cassandra

  • 1. ©2013 DataStax Confidential. Do not distribute without consent. @PatrickMcFadin Patrick McFadin
 Chief Evangelist, DataStax Advanced Cassandra 1
  • 3.
  • 5.
  • 6. Cassandra is not… 6 A Data Ocean or Pond., Lake An In-Memory Database A Key-Value Store A magical database unicorn that farts rainbows
  • 7. 7 When to use… Loose data model (joins, sub-selects) Absolute consistency (aka gotta have ACID) No need to use anything else You’ll miss the long, candle lit dinners with your Oracle rep that always end with “what’s your budget look like this year?” Oracle, MySQL, Postgres or <RDBMS>
  • 8. Uptime is a top priority Unpredictable or high scaling requirements Workload is transactional Willing to put the time or effort into understanding how Cassandra works and how to use it. 8 When to use… Use Oracle when you want to count your money. Use Cassandra when you want to make money. Cassandra
  • 9. Copy n Paste your relational model APACHE CASSANDRA
  • 10.
  • 12. Stick the landing 12 Going to deploy in production! Not sure about this! Done!
  • 13. Topology considerations Replication Strategy CREATE KEYSPACE killrvideo WITH REPLICATION = { 'class' : 'SimpleStrategy', 'replication_factor' : 3 }; Strategy Copies
  • 14. Topology considerations • Default • One data center SimpleStrategy NetworkTopologyStrategy • Use for multi-data center • Just use this always
  • 15. NetworkTopologyStrategy CREATE KEYSPACE Product_Catalog WITH REPLICATION = { 'class' : 'NetworkTopologyStrategy', 'replication_factor' : 3 }; CREATE KEYSPACE EU_Customer_Data WITH REPLICATION = { 'class' : 'NetworkTopologyStrategy', 'eu1' : 3 ‘eu2’ : 3 ‘us1’ : 0 }; Symmetric Asymmetric No copies in the US
  • 16. Application • Closer to customers • No downtime Product_Catalog RF=3 Product_Catalog RF=3 EU_Customer_Data RF=3 EU_Customer_Data RF=0 Product_Catalog RF=3 EU_Customer_Data RF=3
  • 18. Snitches DC1 DC1: RF=3 Node Primary Replica Replica 10.0.0.1 00-25 76-100 51-75 10.0.0.2 26-50 00-25 76-100 10.0.0.3 51-75 26-50 00-25 10.0.0.4 76-100 51-75 26-50 10.0.0.1 00-25 10.0.0.4 76-100 10.0.0.2 26-50 10.0.0.3 51-75 76-100 51-75 00-25 76-100 26-50 00-25 51-75 26-50 Client Where do I place this data? ? Dynamic Snitching Route based on node performance
  • 20. Snitches • Most typically used in production • Absolute placement GossipingPropertyFileSnitch cassandra-rackdc.properties dc=DC1 rack=RAC1
  • 21. Booting a datacenter DC1 DC1: RF=3 Node Primary Replica Replica 10.0.0.1 00-25 76-100 51-75 10.0.0.2 26-50 00-25 76-100 10.0.0.3 51-75 26-50 00-25 10.0.0.4 76-100 51-75 26-50 10.0.0.1 00-25 10.0.0.4 76-100 10.0.0.2 26-50 10.0.0.3 51-75 76-100 51-75 00-25 76-100 26-50 00-25 51-75 26-50 DC2 Pre-check • Use NetworkTopologyStrategy • In cassandra.yaml • auto_bootstrap: false • add seeds from other DC • Set node location for Snitch • GossipingPropertyFileSnitch: cassandra-rackdc.properties • PropertyFileSnitch: cassandra- topology.properties
  • 22. Booting a datacenter DC1 DC1: RF=3 Node Primary Replica Replica 10.0.0.1 00-25 76-100 51-75 10.0.0.2 26-50 00-25 76-100 10.0.0.3 51-75 26-50 00-25 10.0.0.4 76-100 51-75 26-50 10.0.0.1 00-25 10.0.0.4 76-100 10.0.0.2 26-50 10.0.0.3 51-75 76-100 51-75 00-25 76-100 26-50 00-25 51-75 26-50 DC2 10.1.0.1 00-25 10.1.0.4 76-100 10.1.0.2 26-50 10.1.0.3 51-75 76-100 51-75 00-25 76-100 26-50 00-25 51-75 26-50 Node Primary Replica Replica 10.0.0.1 00-25 76-100 51-75 10.0.0.2 26-50 00-25 76-100 10.0.0.3 51-75 26-50 00-25 10.0.0.4 76-100 51-75 26-50 DC2: RF=3 ALTER KEYSPACE
  • 23. Booting a datacenter DC1 DC1: RF=3 Node Primary Replica Replica 10.0.0.1 00-25 76-100 51-75 10.0.0.2 26-50 00-25 76-100 10.0.0.3 51-75 26-50 00-25 10.0.0.4 76-100 51-75 26-50 10.0.0.1 00-25 10.0.0.4 76-100 10.0.0.2 26-50 10.0.0.3 51-75 76-100 51-75 00-25 76-100 26-50 00-25 51-75 26-50 DC2 10.1.0.1 00-25 10.1.0.4 76-100 10.1.0.2 26-50 10.1.0.3 51-75 76-100 51-75 00-25 76-100 26-50 00-25 51-75 26-50 Node Primary Replica Replica 10.0.0.1 00-25 76-100 51-75 10.0.0.2 26-50 00-25 76-100 10.0.0.3 51-75 26-50 00-25 10.0.0.4 76-100 51-75 26-50 DC2: RF=3 nodetool rebuild
  • 25. NoSQL == No Security
  • 26. User Auth Step 1 Turn it on cassandra.yaml authorizer:PasswordAuthorizerAllowAllAuthorizer authenticator:AllowAllAuthenticatorPasswordAuthenticator
  • 27. User Auth cqlsh -u cassandra -p cassandra Step 2 Create users cqlsh> create user dude with password 'manager' superuser; cqlsh> create user worker with password 'newhire'; cqlsh> list users; name | super ----------+------- cassandra | True worker | False dude | True
  • 28. User Auth cqlsh -u cassandra -p cassandra Step 3 Grant permissions cqlsh> create user ro_user with password '1234567'; cqlsh> grant all on killrvideo.user to dude; cqlsh> grant select on killrvideo.user to ro_user;
  • 30. Prepared Statements • Built for speed an efficiency
  • 31. How they work: Prepare SELECT * FROM user WHERE id = ? 10.0.0.1 00-25 10.0.0.4 76-100 10.0.0.2 26-50 10.0.0.3 51-75 Client Prepare Parsed Hashed Cached Prepared Statement
  • 32. How they work: Bind id = 1 + PreparedStatement Hash 10.0.0.1 00-25 10.0.0.4 76-100 10.0.0.2 26-50 10.0.0.3 51-75 Client Bind & Execute Combine Pre-parsed Query and Variable Execute
  • 34. How to Prepare(Statements) PreparedStatement userSelect = session.prepare(“SELECT * FROM user WHERE id = ?”); BoundStatement userSelectStatement = new BoundStatement(userSelect); session.execute(userSelectStatement.bind(1)); prepared_stmt = session.prepare (“SELECT * FROM user WHERE id = ?”) bound_stmt = prepared_stmt.bind([1]) session.execute(bound_stmt) Java Python
  • 35. Don’t do this for (int i = 1; i < 100; i++) { PreparedStatement userSelect = session.prepare(“SELECT * FROM user WHERE id = ?”); BoundStatement userSelectStatement = new BoundStatement(userSelect); session.execute(userSelectStatement.bind(1)); }
  • 36. Execute vs Execute Async • Very subtle difference • Blocking vs non-blocking call VS
  • 37. Async • Request pipelining • One connection for requests • Responses return whenever
  • 38. Async for (…) { future = executeAsync(statement) } 10.0.0.1 00-25 10.0.0.4 76-100 10.0.0.2 26-50 10.0.0.3 51-75 Client Do something for (…) { result = future.get } Block
  • 39. Batch vs Execute Async VS (Potentially)
  • 40. Load Balancing Policies cluster = Cluster .builder() .addContactPoint("192.168.0.30") .withQueryOptions(new QueryOptions().setConsistencyLevel(ConsistencyLevel.ONE) .withRetryPolicy(DefaultRetryPolicy.INSTANCE) .withLoadBalancingPolicy(new TokenAwarePolicy(new DCAwareRoundRobinPolicy())) .build(); session = cluster.connect("demo");
  • 41. Data Locality DC1 DC1: RF=3 Node Primary Replica Replica 10.0.0.1 00-25 76-100 51-75 10.0.0.2 26-50 00-25 76-100 10.0.0.3 51-75 26-50 00-25 10.0.0.4 76-100 51-75 26-50 10.0.0.1 00-25 10.0.0.4 76-100 10.0.0.2 26-50 10.0.0.3 51-75 76-100 51-75 00-25 76-100 26-50 00-25 51-75 26-50 Client Read partition 15 DC2 10.1.0.1 00-25 10.1.0.4 76-100 10.1.0.2 26-50 10.1.0.3 51-75 76-100 51-75 00-25 76-100 26-50 00-25 51-75 26-50 Node Primary Replica Replica 10.0.0.1 00-25 76-100 51-75 10.0.0.2 26-50 00-25 76-100 10.0.0.3 51-75 26-50 00-25 10.0.0.4 76-100 51-75 26-50 DC2: RF=3 Client Read partition 15
  • 42. Batch (Logged) • All statements collected on client • Sent in one shot • All done on 1 node Batch is accepted All actions are logged on two replicas Statements executed in sequence Results are collected and returned
  • 43. Batches: The good • Great for denormalized inserts/updates // Looking from the video side to many users CREATE TABLE comments_by_video ( videoid uuid, commentid timeuuid, userid uuid, comment text, PRIMARY KEY (videoid, commentid) ) WITH CLUSTERING ORDER BY (commentid DESC); // looking from the user side to many videos CREATE TABLE comments_by_user ( userid uuid, commentid timeuuid, videoid uuid, comment text, PRIMARY KEY (userid, commentid) ) WITH CLUSTERING ORDER BY (commentid DESC);
  • 44. Batches: The good • Both inserts are run • On failure, the batch log will replay BEGIN BATCH INSERT INTO comments_by_video (videoid, userid, commentid, comment) VALUES (99051fe9-6a9c-46c2-b949-38ef78858dd0,d0f60aa8-54a9-4840-b70c-fe562b68842b,now(), 'Worst. Video. Ever.') INSERT INTO comments_by_video (videoid, userid, commentid, comment) VALUES (99051fe9-6a9c-46c2-b949-38ef78858dd0,d0f60aa8-54a9-4840-b70c-fe562b68842b,now(), 'Worst. Video. Ever.') APPLY BATCH;
  • 45. Batches: The bad “I was doing a load test and nodes started blinking offline” “Were you using a batch by any chance?” “Why yes I was! How did you know?” “How big was each batch?” “1000 inserts each”
  • 46. Batches: The bad BEGIN BATCH 1000 inserts APPLY BATCH; 10.0.0.1 00-25 10.0.0.4 76-100 10.0.0.2 26-50 10.0.0.3 51-75 Client
  • 47. Batches: The rules • Keep them small and for atomicity CASSANDRA-6487 - Warn on large batches (5Kb default) CASSANDRA-8011 - Fail on large batches (50Kb default)
  • 48. The alternative BEGIN BATCH 1000 inserts APPLY BATCH; while() { future = session.executeAsync(statement) } Instead of: Do this:
  • 49. Old Row cache: The problem • Reads an entire storage row of data ID = 1 Partition Key (Storage Row Key) 2014-09-08 12:00:00 : name SFO 2014-09-08 12:00:00 : temp 63.4 2014-09-08 12:01:00 : name SFO 2014-09-08 12:00:00 : temp 63.9 2014-09-08 12:02:00 : name SFO 2014-09-08 12:00:00 : temp 64.0 Need this Caches this
  • 50. New Row Cache: The solution • Stores just a few CQL rows ID = 1 Partition Key (Storage Row Key) 2014-09-08 12:00:00 : name SFO 2014-09-08 12:00:00 : temp 63.4 2014-09-08 12:01:00 : name SFO 2014-09-08 12:00:00 : temp 63.9 2014-09-08 12:02:00 : name SFO 2014-09-08 12:00:00 : temp 64.0 Need this Caches this
  • 51. Using row cache CREATE TABLE user_search_history_with_cache ( id int, search_time timestamp, search_text text, search_results int, PRIMARY KEY (id, search_time) ) WITH CLUSTERING ORDER BY (search_time DESC) AND caching = { 'keys' : 'ALL', 'rows_per_partition' : '20' };
  • 53. Go make something awesome
  • 54. Thank you! Bring the questions Follow me on twitter @PatrickMcFadin