This document discusses Cassandra use cases and anti-patterns. It describes queue-like designs, intensive updates on the same column, and designing around a dynamic schema as anti-patterns that can lead to failures. Rate limiting, fraud prevention, and account validation are provided as examples of good use cases. Key-value modeling, clustering, compaction strategies, and time-to-live features are also overviewed.
6. Last Write Win (LWW)!
INSERT INTO users(login, name, age) VALUES(‘jdoe’, ‘John DOE’, 33);
@doanduyhai
6
jdoe
age
name
33 John DOE
#partition
7. Last Write Win (LWW)!
@doanduyhai
jdoe
age (t1) name (t1)
33 John DOE
7
INSERT INTO users(login, name, age) VALUES(‘jdoe’, ‘John DOE’, 33);
auto-generated timestamp (μs)
.
8. Last Write Win (LWW)!
@doanduyhai
8
UPDATE users SET age = 34 WHERE login = jdoe;
jdoe
SSTable1 SSTable2
age (t1) name (t1)
33 John DOE
jdoe
age (t2)
34
9. Last Write Win (LWW)!
@doanduyhai
9
DELETE age FROM users WHERE login = jdoe;
tombstone
SSTable1 SSTable2 SSTable3
jdoe
age (t3)
ý
jdoe
age (t1) name (t1)
33 John DOE
jdoe
age (t2)
34
10. Last Write Win (LWW)!
@doanduyhai
10
SELECT age FROM users WHERE login = jdoe;
? ? ?
SSTable1 SSTable2 SSTable3
jdoe
age (t3)
ý
jdoe
age (t1) name (t1)
33 John DOE
jdoe
age (t2)
34
11. Last Write Win (LWW)!
@doanduyhai
11
SELECT age FROM users WHERE login = jdoe;
✕ ✕ ✓
SSTable1 SSTable2 SSTable3
jdoe
age (t3)
ý
jdoe
age (t1) name (t1)
33 John DOE
jdoe
age (t2)
34
12. Compaction!
@doanduyhai
12
SSTable1 SSTable2 SSTable3
jdoe
age (t3)
ý
jdoe
age (t1) name (t1)
33 John DOE
jdoe
age (t2)
34
New SSTable
jdoe
age (t3) name (t1)
ý John DOE
30. Queue-like designs!
Read cursor. Next read will give {A, E}
@doanduyhai
30
Solution: event-sourcing
• write ahead, never delete
• read = move a cursor forward (or backward in time for history)
A B C D A E
Write cursor
31. CQL null semantics!
@doanduyhai
31
Reading null value means
• value does not exist (has never bean created)
• value deleted (tombstone)
SELECT age FROM users WHERE login = jdoe; à NULL
32. CQL null semantics!
@doanduyhai
32
Writing null means
• delete value (creating tombstone)
• even though it does not exist
UPDATE users SET age = NULL WHERE login = jdoe;
33. CQL null semantics!
@doanduyhai
33
Seen in production: prepared statement
UPDATE users SET
age = ?,
…
geo_location = ?,
mood = ?,
…
WHERE login = ?;
34. CQL null semantics!
@doanduyhai
34
Seen in production: bound statement
preparedStatement.bind(33, …, null, null, null, …);
null ☞ tombstone creation on each update …
jdoe
age name geo_loc mood status
33 John DOE ý ý ý
36. Intensive update!
@doanduyhai
36
Context
• small start-up
• cloud-based video recording & alarm
• internet of things (sensor)
• 10 updates/sec for some sensors
37. Intensive update on same column!
@doanduyhai
37
Data model
sensor_id
value
45.0034
CREATE TABLE sensor_data (
sensor_id long,
value double,
PRIMARY KEY(sensor_id));
38. Intensive update on same column!
UPDATE sensor_data SET value = 45.0034 WHERE sensor_id = …;
UPDATE sensor_data SET value = 47.4182 WHERE sensor_id = …;
UPDATE sensor_data SET value = 48.0300 WHERE sensor_id = …;
@doanduyhai
38
Updates
sensor_id
value (t1)
45.0034
sensor_id
value (t13)
47.4182
sensor_id
value (t36)
48.0300
39. Intensive update on same column!
@doanduyhai
39
Read
SELECT sensor_value from sensor_data WHERE sensor_id = …;
read N physical columns, only 1 useful … (until compaction)
sensor_id
value (t1)
45.0034
sensor_id
value (t13)
47.4182
sensor_id
value (t36)
48.0300
41. Intensive update on same column!
@doanduyhai
41
Solution 1: leveled compaction! (if your I/O can keep up)
sensor_id
value (t1)
45.0034
sensor_id
value (t13)
47.4182
sensor_id
value (t36)
48.0300
sensor_id
value (t36)
48.0300
42. Intensive update on same column!
@doanduyhai
42
Solution 2: reversed timeseries & DateTiered compaction strategy
CREATE TABLE sensor_data (
sensor_id long,
date timestamp,
value double,
PRIMARY KEY((sensor_id), date))
WITH CLUSTERING ORDER (date DESC);
43. Intensive update on same column!
SELECT sensor_value FROM sensor_data WHERE sensor_id = … LIMIT 1;
@doanduyhai
43
sensor_id
date3(t3)
date2(t2)
date1(t1)
Data cleaning by configuration the strategy (base_time_seconds)
...
48.0300 47.4182 45.0034 …
44. Design around dynamic schema!
@doanduyhai
44
Customer emergency call
• 3 nodes cluster almost full
• impossible to scale out
• 4th node in JOINING state for 1 week
• disk space is filling up, production at risk!
45. Design around dynamic schema!
@doanduyhai
45
After investigation
• 4th node in JOINING state because streaming is stalled
• NPE in logs
46. Design around dynamic schema!
@doanduyhai
46
After investigation
• 4th node in JOINING state because streaming is stalled
• NPE in logs
Cassandra source-code to the rescue
47. Design around dynamic schema!
@doanduyhai
47
public class CompressedStreamReader extends StreamReader
{
…
@Override
public SSTableWriter read(ReadableByteChannel channel) throws IOException
{
…
Pair<String, String> kscf = Schema.instance.getCF(cfId);
ColumnFamilyStore cfs = Keyspace.open(kscf.left).getColumnFamilyStore(kscf.right);
NPE here
48. Design around dynamic schema!
@doanduyhai
48
The truth is
• the devs dynamically drop & recreate table every day
• dynamic schema is in the core of their design
Example:
DROP TABLE catalog_127_20140613;
CREATE TABLE catalog_127_20140614( … );
52. Design around dynamic schema!
@doanduyhai
52
Nutshell
• dynamic schema change as normal prod operation is not
recommended
• schema AND topology change at the same time is an anti-pattern
65. Rate limiting!
@doanduyhai
65
Solution
• premium phone number ☞ Google libphonenumber
• massive hack ☞ rate limiting with Cassandra
66. Cassandra Time To Live!
@doanduyhai
66
Time to live
• built-in feature
• insert data with a TTL in sec
• expires server-side automatically
• ☞ use as sliding-window
67. Rate limiting in action!
@doanduyhai
67
Implementation
• threshold = max 3 reset password per sliding 24h per
user
68. Rate limiting in action!
@doanduyhai
68
Implementation
• when /password/reset called
• check threshold
• reached ☞ error message/ignore
• not reached ☞ log the attempt with TTL = 86400
70. Anti Fraud!
@doanduyhai
70
Real story
• many special offers available
• 30 mins international calls (50 countries)
• unlimited land-line calls to 5 countries
• …
71. Anti Fraud!
@doanduyhai
71
Real story
• each offer has a duration (week/month/year)
• only one offer active at a time
72. Anti Fraud!
@doanduyhai
72
Cassandra TTL
• when granting new offer
INSERT INTO user_special_offer(login, offer_code, …)
VALUES(‘jdoe’, ’30_mins_international’,…)
IF NOT EXISTS
USING TTL <offer_duration>;
74. Account Validation!
@doanduyhai
74
Requirement
• user creates new account
• sends sms/email link with token to validate account
• 10 days to validate
75. Account Validation!
@doanduyhai
75
How to ?
• create account with 10 days TTL
INSERT INTO users(login, name, age)
VALUES(‘jdoe’, ‘John DOE’, 33)
USING TTL 864000;
76. Account Validation!
@doanduyhai
76
How to ?
• create random token for validation with 10 days TTL
INSERT INTO account_validation(token, login, name, age)
VALUES(‘A0F83E63DB935465CE73DFE…’, ‘jdoe’, ‘John DOE’, 33)
USING TTL 864000;
77. Account Validation!
@doanduyhai
77
On token validation
• check token exist & retrieve user details
SELECT login, name, age FROM account_validation
WHERE token = ‘A0F83E63DB935465CE73DFE…’;
• re-insert durably user details without TTL
INSERT INTO users(login, name, age) VALUES(‘jdoe’, ‘John DOE’, 33);