Who wants to be a Cassandra Millionaire

Cassandra Installation to Optimization
40-minutes of best practice and resources
@VictorFAnjos

© Mark E. Damon - All Rights Reserved
15
14
13
12
11
10
9
8
7
6
5
4
3
2
1
1 Million
500,000
250,000
125,000
64,000
32,000
16,000
8,000
4,000
2,000
1,000
500
300
200
100
Welcome to
Who Wants to
perform 1M ops/s
50:50
@VictorFAnjos

15
14
13
12
11
10
9
8
7
6
5
4
3
2
1
1 Million
500,000
250,000
125,000
64,000
32,000
16,000
8,000
4,000
2,000
1,000
500
300
200
100
@VictorFAnjos

5© 2015. All Rights Reserved.
A: NAS / SAN
C: DAS SATA
B: SSD
D: DAS SCSI
50:50
15
14
13
12
11
10
9
8
7
6
5
4
3
2
1
1 Million
500,000
250,000
125,000
64,000
32,000
16,000
8,000
4,000
2,000
1,000
500
300
200
100
This storage
medium allows for
best performance.
@VictorFAnjos

@VictorFAnjos
A: NAS / SAN
C: DAS SATA
B: SSD
D: DAS SCSI
50:50
15
14
13
12
11
10
9
8
7
6
5
4
3
2
1
1 Million
500,000
250,000
125,000
64,000
32,000
16,000
8,000
4,000
2,000
1,000
500
300
200
100
@VictorFAnjos
This storage
medium allows for
best performance.

Installation and considerations
how to store the datastore
Storage Area Network Solid State Drive
@VictorFAnjos

Local (DAS), iSCSI, Fiber Channel
@VictorFAnjos
● AVOID network storage like the plague
● Direct Attached Storage FTW
● Disk latency is a HUGE deal for performance

9© 2015. All Rights Reserved. 9
@VictorFAnjos
SATA/SAS DAS
PCIe/NVMe DAS

@VictorFAnjos

@VictorFAnjos
A: ZFS
C: Ext4
B: Btrfs
D: F2FS
50:50
15
14
13
12
11
10
9
8
7
6
5
4
3
2
1
1 Million
500,000
250,000
125,000
64,000
32,000
16,000
8,000
4,000
2,000
1,000
500
300
200
100
When using SSDs,
this filesystem type
is best.
@VictorFAnjos

Congratulations!
You’ve Reached
the 1,000 ops/s
Milestone!
Congratulations!Congratulations!
@VictorFAnjos

i can’t believe it’s not btrfs
@VictorFAnjos
● easiest to use ext4 (it’s on most linux
distros), but F2FS get 5-10% gains in write
performance
● if NOT using F2FS, make sure to TRIM
● multiple disks → use RAID0

@VictorFAnjos
A: 0
C: Equal to HEAP
B: ½ of HEAP
D: EQUAL TO RAM
50:50
15
14
13
12
11
10
9
8
7
6
5
4
3
2
1
1 Million
500,000
250,000
125,000
64,000
32,000
16,000
8,000
4,000
2,000
1,000
500
300
200
100
This is the
sweetspot for SWAP
when using C*
@VictorFAnjos

to swap or not to swap
@VictorFAnjos

@VictorFAnjos
A: 64G
C: 16G
B: 32G
D: 8G
50:50
15
14
13
12
11
10
9
8
7
6
5
4
3
2
1
1 Million
500,000
250,000
125,000
64,000
32,000
16,000
8,000
4,000
2,000
1,000
500
300
200
100
Having 64G of RAM
means you should
optimize to have
___G of HEAP.
@VictorFAnjos

@VictorFAnjos
A: 64G
C: 16G
B: 32G
D: 8G
50:50
15
14
13
12
11
10
9
8
7
6
5
4
3
2
1
1 Million
500,000
250,000
125,000
64,000
32,000
16,000
8,000
4,000
2,000
1,000
500
300
200
100
@VictorFAnjos
Having 64G of RAM
means you should
optimize to have
___G of HEAP.

how much heap?
@VictorFAnjos
http://docs.datastax.com/en/cassandra/1.2/cassandra/operations/ops_tune_jvm_c.html

@VictorFAnjos
A: EC2Snitch
C: Simple Snitch
B: Dynamic Snitch
D: Property File Snitch
50:50
15
14
13
12
11
10
9
8
7
6
5
4
3
2
1
1 Million
500,000
250,000
125,000
64,000
32,000
16,000
8,000
4,000
2,000
1,000
500
300
200
100
Definitely DO NOT
use this snitch in
Multi-DC
environments.
@VictorFAnjos

@VictorFAnjos
A: EC2Snitch
C: Simple Snitch
B: Dynamic Snitch
D: Property File Snitch
50:50
15
14
13
12
11
10
9
8
7
6
5
4
3
2
1
1 Million
500,000
250,000
125,000
64,000
32,000
16,000
8,000
4,000
2,000
1,000
500
300
200
100
@VictorFAnjos
Definitely DO NOT
use this snitch in
Multi-DC
environments.

son of a snitch
@VictorFAnjos

@VictorFAnjos
A: Synchronous AND Full Queries
C: Synchronous AND Prepared Statements
B: Asynchronous AND Prepared Statements
D: Asynchronous AND Full Queries
50:50
15
14
13
12
11
10
9
8
7
6
5
4
3
2
1
1 Million
500,000
250,000
125,000
64,000
32,000
16,000
8,000
4,000
2,000
1,000
500
300
200
100
To reduce latency
and wire time to my
app, I should opt
for.
@VictorFAnjos

Achieving performance through code/drivers
should I stay or should I go
@VictorFAnjos
● Client writes to any Cassandra node
● Coordinator node replicates to other nodes
(in local and remote Data Center)
● Local write acks returned to coordinator
● Client gets ack when enough total nodes
are committed
● Data written to internal commit log disks
● When data arrives, remote node replicates
data
MULTI DC
● Ack direct to source region coordinator
● Remote copies written to commit log disks
lf a node or region goes offline, hinted
handoff completes the write when the
node comes back up (as long as there are
enough nodes to satisfy consistency level).

@VictorFAnjos
Prepare ONCE...
Bind and Execute multiple times.

@VictorFAnjos

@VictorFAnjos
A: 1 / 1 = 1
C: 2 * 1 = 2
B: 2 / 1 = 2
D: 2 / 2 + 1 = 2
50:50
15
14
13
12
11
10
9
8
7
6
5
4
3
2
1
1 Million
500,000
250,000
125,000
64,000
32,000
16,000
8,000
4,000
2,000
1,000
500
300
200
100
With RF=2 and
CL=Quorum, operations
failed when 1 node went
down because of this.
@VictorFAnjos

Congratulations!
You’ve Reached
the 32,000 ops/s
Milestone!
Congratulations!Congratulations!
@VictorFAnjos

when friends aren’t enough
@VictorFAnjos
Replication Factor = 3
Insert into a cluster of size 6 with
consistency Quorum
Two nodes in token range must be
present for write to succeed

when friends aren’t enough
@VictorFAnjos
What happens now?
Cannot achieve consistency level QUORUM
Nodes in partition key DOWN

@VictorFAnjos
A: Truth table
C: CAP Theorem
B: Brewer’s Theorem
D: Entropy
50:50
15
14
13
12
11
10
9
8
7
6
5
4
3
2
1
1 Million
500,000
250,000
125,000
64,000
32,000
16,000
8,000
4,000
2,000
1,000
500
300
200
100
This mathematical and
CS concept helps when
data modeling for query
optimization.
@VictorFAnjos

@VictorFAnjos
A: Truth table
C: CAP Theorem
B: Brewer’s Theorem
D: Entropy
50:50
15
14
13
12
11
10
9
8
7
6
5
4
3
2
1
1 Million
500,000
250,000
125,000
64,000
32,000
16,000
8,000
4,000
2,000
1,000
500
300
200
100
@VictorFAnjos
This mathematical and
CS concept helps when
data modeling for query
optimization.

Data modelling, CQLSH and more
the truth shall set you free
@VictorFAnjos
Motivated by CS, Math, Engineering
Allows for creating building blocks
that yield a single output
More complex truth tables can arise

the truth shall set you free
@VictorFAnjos
How about searching for username?
And what about full_name?
user_stream
← ← ← Partition Key → → →
user_id username full_name
1 0
0
0 1
0
0 0
1

@VictorFAnjos
A: Reads / Batches
C: Writes / Deletes
B: Writes / Batches
D: Reads / Deletes
50:50
15
14
13
12
11
10
9
8
7
6
5
4
3
2
1
1 Million
500,000
250,000
125,000
64,000
32,000
16,000
8,000
4,000
2,000
1,000
500
300
200
100
A shift in paradigms,
what should you
maximize and reduce for
good performance.
@VictorFAnjos

@VictorFAnjos
A: Reads / Batches
C: Writes / Deletes
B: Writes / Batches
D: Reads / Deletes
50:50
15
14
13
12
11
10
9
8
7
6
5
4
3
2
1
1 Million
500,000
250,000
125,000
64,000
32,000
16,000
8,000
4,000
2,000
1,000
500
300
200
100
@VictorFAnjos
A shift in paradigms,
what should you
maximize and reduce for
good performance.

do the write thing
@VictorFAnjos

do the write thing
@VictorFAnjos
memtable --- < 100ns
commit log --- ~ 1 ms
DELETES / TTL cause compactions

do the write thing
@VictorFAnjos

@VictorFAnjos
A: ACID
C: Rollback
B: Vector
D: Sharding
50:50
15
14
13
12
11
10
9
8
7
6
5
4
3
2
1
1 Million
500,000
250,000
125,000
64,000
32,000
16,000
8,000
4,000
2,000
1,000
500
300
200
100
To not hit a 2B record
limit (per row), this
RDBMS borrowed term
can still makes sense.
@VictorFAnjos

sit on this and rotate
@VictorFAnjos

@VictorFAnjos
A: Batches
C: Secondary Indexes
B: Synchronous
D: MySQL
50:50
15
14
13
12
11
10
9
8
7
6
5
4
3
2
1
1 Million
500,000
250,000
125,000
64,000
32,000
16,000
8,000
4,000
2,000
1,000
500
300
200
100
Many say to use
sparingly, I would say,
avoid like the plague.
@VictorFAnjos

@VictorFAnjos
A: Batches
C: Secondary Indexes
B: Synchronous
D: MySQL
50:50
15
14
13
12
11
10
9
8
7
6
5
4
3
2
1
1 Million
500,000
250,000
125,000
64,000
32,000
16,000
8,000
4,000
2,000
1,000
500
300
200
100
@VictorFAnjos
Many say to use
sparingly, I would say,
avoid like the plague.

Performance must-haves
never be second best
@VictorFAnjos
writes are distributed among the cluster
each partition key refers to one exact
position in which to get a row
but what do we do when we don’t have exactly
the right type of index to specify a query
CREATE TABLE users (
user varchar,
email varchar,
state varchar,
PRIMARY KEY (user));
-- OPTION 1 : create an index
CREATE INDEX idxUBS on users (state);
-- OPTION 2 : create another table (store data twice)
CREATE TABLE usersByState (
state varchar,
user varchar,
PRIMARY KEY (state, user));

@VictorFAnjos
A: UDT
C: JSON
B: Lightweight Transactions
D: Hinted handoff
50:50
15
14
13
12
11
10
9
8
7
6
5
4
3
2
1
1 Million
500,000
250,000
125,000
64,000
32,000
16,000
8,000
4,000
2,000
1,000
500
300
200
100
This recent addition to
C* now helps with ACID
like transactions, at a bit
of a performance hit.
@VictorFAnjos

slimfast agreement
@VictorFAnjos
Prepares a proposal that is sent to a number of Acceptors.
Waits on a an acknowledgement (in form of promise) from Acceptors.
Sends accept message to Quorum of Acceptors with new value to commit.
Returns success? completion to client.
Determines if proposal is newer than what it has seen.
Acknowledges/agree with its own highest proposal value seen AND the
current value (of what is to be set).
Receive message to commit new value.
Accept and return on successful commit of value.

@VictorFAnjos
slimfast agreement

Who wants to be a Cassandra Millionaire

Recommended

Recommended

More Related Content

Similar to Who wants to be a Cassandra Millionaire

Similar to Who wants to be a Cassandra Millionaire (20)

More from Victor Anjos

More from Victor Anjos (6)

Recently uploaded

Recently uploaded (20)

Who wants to be a Cassandra Millionaire