Recommended
More Related Content
Similar to Cassandra SF 2012 - Technical Deep Dive: query performance
Similar to Cassandra SF 2012 - Technical Deep Dive: query performance (20)
More from aaronmorton
More from aaronmorton (20)
Recently uploaded
Recently uploaded (20)
Cassandra SF 2012 - Technical Deep Dive: query performance
- 1. TECHNICAL DEEP DIVE: QUERY PERFORMANCE CASSANDRA SF 2012 Aaron Morton, Apache Cassandra Committer @aaronmorton www.thelastpickle.com Licensed under a Creative Commons Attribution-NonCommercial 3.0 New Zealand License
- 2. Today. Write Path Read Path
- 3. 7 Microseconds of Terror.
- 4. Standard write path... 1. Append to Commit Log 2. Merge with Memtable
- 5. Blast through the thin Martian atmosphere! (Also time to synchronise around the commit log.)
- 6. Write path details... Acquire read lock from server wide read-write switchLock.
- 7. Memtable flush path acquires the write lock from switchLock.
- 8. Flush will block if memtable_flush_queue_s ize Memtables are waiting to flush.
- 9. memtable_flush_queue_size test... m1.xlarge Cassandra node m1.xlarge client node 1 CF with 6 Secondary Indexes 1 Client Thread 10,000 Inserts, 100 Columns per Row 1100 bytes per Column
- 10. memtable_flush_queue_size test... create column family WithIndex with comparator = AsciiType and default_validation_class = AsciiType and key_validation_class = AsciiType and max_compaction_threshold = 0 and min_compaction_threshold = 0 and compression_options = null and column_metadata [{ column_name : 0000000000, validation_class : AsciiType, index_type : 0, index_name : 0000000000},
- 11. CF write latency and memtable_flush_queue_size... memtable_flush_queue_size=7 memtable_flush_queue_size=1 1,200 900 Latency Microseconds 600 300 0 85th 95th 99th 100th
- 12. Request latency and memtable_flush_queue_size... memtable_flush_queue_size=7 memtable_flush_queue_size=1 5,000,000 3,750,000 Latecy Microseconds 2,500,000 1,250,000 0 85th 95th 99th 100th
- 13. Deploy the Hypersonic parachute! (Also time to write to the CommitLog.)
- 14. Commit Log... Write to CommitLog if Keyspace wide durable_writes is enabled. (Enabled by default.)
- 15. durable_writes test... 10,000 Inserts, 50 Columns per Row 50 bytes per Column
- 16. Request latency and durable_writes (1 client)... enabled disabled 7,000 5,250 Latency Microseconds 3,500 1,750 0 85th 95th 99th
- 17. Request latency and durable_writes (10 clients)... enabled disabled 30,000 22,500 Latency Microseconds 15,000 7,500 0 85th 95th 99th
- 18. Request latency and durable_writes (20 clients)... enabled disabled 90,000 67,500 Latency Microseconds 45,000 22,500 0 85th 95th 99th
- 19. CommitLog has either periodic or batch file syncing. (Periodic with fsync every 10 seconds is default.)
- 20. periodic commit log adds mutation to queue then acknowledges. Commit Log is appended to by a single thread, sync is called every commitlog_sync_period_in_ms.
- 21. CommitLog tests... 10,000 Inserts, 50 Columns per Row 50 bytes per Column
- 22. Request latency and commitlog_sync_period_in_ms... 10,000 ms 10 ms 220 208 Latecy Microseconds 195 183 170 85th 95th 99th
- 23. batch commit log adds mutation to queue and waits before acknowledging. Writer thread processes mutations for commitlog_sync_batch_window_in_ms duration, then syncs, then signals.
- 24. Request latency and commitlog_sync_batch_window_in_ms... 50 ms 0 ms 800 750 Latecy Microseconds 700 650 600 85th 95th 99th
- 25. Request latency comparing periodic and batch sync... periodic batch 800 600 Latecy Microseconds 400 200 0 85th 95th 99th
- 26. Engage the rocket powered sky crane! (Also time to merge the mutation with the current Memtable.)
- 27. Merge mutation... Row level Isolation provided via SnapTree. (https://github.com/nbronson/snaptree)
- 28. 00 REM Cassandra for C64 05 REM Clone row_cols into my_cols 10 GOSUB 1000 20 FOR col = write.first_col TO write.last_col 25 REM Add or Reconcile col with my_cols 30 GOSUB 2000 40 IF my_cols != row_cols THEN GOTO 05 50 NEXT col 55 REM Atomic swap row_cols with my_cols 60 GOSUB 3000 70 IF swapped_cols = FALSE THEN GOTO 05
- 29. Row concurrency tests... 10,000 Columns per Row 50 bytes per Column 50 Columns per Insert
- 30. CF Write Latency and row concurrency (10 clients)... different rows single row 2,000 1,500 Latecy Microseconds 1,000 500 0 85th 95th 99th
- 31. Write path with KEYS secondary indexes... 1. Append to Commit Log 2. Read modified Index Columns 3. Merge with Memtable 4. Update Secondary Indexes
- 32. Secondary Indexes... synchronized access to indexed rows. (Keyspace wide)
- 33. Index concurrency tests... CF with 2 Indexes 10,000 Inserts 6 Columns per Row 35 bytes per Column Alternating column values
- 34. Request latency and index concurrency (10 clients)... different rows single row 4,000 3,000 Latecy Microseconds 2,000 1,000 0 85th 95th 99th
- 35. Read modified indexed columns or all for row deletion.
- 36. Insert or delete changes to index rows.
- 37. Index tests... 10,000 Inserts 50 Columns per Row 50 bytes per Column
- 38. Request latency and secondary indexes... no indexes six indexes 3,000 2,250 Latecy Microseconds 1,500 750 0 85th 95th 99th
- 39. Today Write Path Read Path
- 40. Two read paths... SliceByNamesReadCommand SliceFromReadCommand
- 41. Locating row start per SSTable... 1. Check BloomFilter 2. Read KeyCache 3. Read Index Samples, seek and partial scan -Index.db (Step 3 only used if KeyCache lookup missed.)
- 42. Column Family bloom_filter_fp_chance controls BloomFilter effectiveness.
- 43. bloom_filter_fp_chance tests... 1,000,000 Rows 50 Columns per Row 50 bytes per Column commitlog_total_space_in_mb: 1 Read random 10% of rows.
- 44. CF read latency and bloom_filter_fp_chance... default 0.000744. 0.1 7,000 5,250 Latecy Microseconds 3,500 1,750 0 85th 95th 99th
- 45. SSTables per read and bloom_filter_fp_chance (14 SStables)... default 0.000744. 0.1 1 1 SSTables per Read 1 0 0 85th 95th 99th
- 46. Bloom Filter False Ratio and bloom_filter_fp_chance... default 0.000744. 0.1 1 1 False Positive Ratio 0 0 0
- 47. key_cache_size_in_mb controls the size of the key cache.
- 48. key_cache_size_in_mb tests... 10,000 Rows 50 Columns per Row 50 bytes per Column Read all Rows
- 49. CF read latency and key_cache_size_in_mb... default (100MB) 100% Hit Rate disabled 300 225 Latecy Microseconds 150 75 0 85th 95th 99th
- 50. index_interval controls the size of the -Index.db sampling.
- 51. index_interval tests... 100,000 Rows 50 Columns per Row 50 bytes per Column key_cache_size_in_mb: 0 Read 1 Column from random 10% of Rows
- 52. CF read latency and index_interval... index_interval=128 (default) index_interval=512 20,000 15,000 Latecy Microseconds 10,000 5,000 0 85th 95th 99th
- 53. Row Cache removes all disk IO.
- 54. row_cache_size_in_mb controls the size of Row Cache.
- 55. row_cache_size_in_mb tests... 100,000 Rows 50 Columns per Row 50 bytes per Column Read all Rows
- 56. CF read latency and row_cache_size_in_mb... row_cache_size_in_mb=0 and key_cache_size_in_mb=100mb row_cache_size_in_mb=100mb and key_cache_size_in_mb=0 260 195 Latecy Microseconds 130 65 0 85th 95th 99th
- 57. Slice by Name (excluding Counter Columns)... 1. Order SSTables by maxTimestamp. 2. Break if SSTable covered by previous Row Tombstone. 3. Remove irrelevant Columns from query. 4. Eagerly read Columns from SSTable. 5. Hoist Columns into current Size Tier.
- 58. When reading by Column name... A query on a wide row must read the entire Column Index.
- 59. Column Index tests... Read first Column by name from 1,200 Columns. Read first Column by name from 1,000,000 Columns.
- 60. CF read latency and Column Index... First Column from 1,200 First Column from 1,000,000 6,000 4,500 Latecy Microseconds 3,000 1,500 0 85th 95th 99th
- 61. When reading by Column name... A narrow query on a wide row performs better.
- 62. Name Locality tests... 1,000,000 Columns 50 bytes per Column Read 100 Columns from middle of row. Read 100 Columns from spread across row.
- 63. CF read latency and name locality... Adjacent Columns Spread Columns 200,000 150,000 Latecy Microseconds 100,000 50,000 0 85th 95th 99th
- 64. Slice... 1. Skip if SSTable covered by previous Row Tombstone. 2. Read and merge Columns from SSTable. (May be eager or lazy reading)
- 65. When slicing... A query using natural Column order without a start has the highest performance.
- 66. Start position tests... 1,000,000 Columns 50 bytes per Column Read first 100 Columns without start. Read first 100 Columns with start.
- 67. CF read latency and start position... Without start position With start position 40,000 30,000 Latecy Microseconds 20,000 10,000 0 85th 95th 99th
- 68. When slicing... Any start column requires the Column index to be read.
- 69. Start offset tests... 1,000,000 Columns 50 bytes per Column Read first 100 Columns with start. Read middle 100 Columns with start.
- 70. CF read latency and start offset... First MIddle 40,000 30,000 Latecy Microseconds 20,000 10,000 0 85th 95th 99th
- 71. When slicing... Reverse requires the Column index to be read.
- 72. Start offset tests... 1,000,000 Columns 50 bytes per Column Read first 100 Columns without start. Read last 100 Columns with reversed.
- 73. CF read latency and reversed... Forward Reversed 40,000 30,000 Latecy Microseconds 20,000 10,000 0 85th 95th 99th
- 74. Aaron Morton @aaronmorton www.thelastpickle.com Licensed under a Creative Commons Attribution-NonCommercial 3.0 New Zealand License
Editor's Notes
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n