®

Fractal Tree Indexes
Theory to Practice
Percona Live London 2013
Tim Callaghan, Tokutek
tim@tokutek.com
@tmcallaghan
®
...
Ever seen this?

IO Utilization Graph, performance is IO limited

®

Tuesday, November 12, 13
Who is Tokutek?

Tokutek builds highperformance database
software!
TokuDB - storage engine for
MySQL and MariaDB
TokuMX - ...
Who am I?

• 17 year database consumer
• schema design, development, deployment
• database administration + infrastructure...
Housekeeping

• Feedback is important to me
• Ideas for Webinars or Presentations?
• Who’s using MongoDB?
• Anyone using T...
Agenda

• Why Fractal Tree indexes are cool
• What they enable in MySQL® (TokuDB)
• What they enable in MongoDB® (TokuMX)
...
Indexing:
B-trees and
Fractal Tree Indexes

®

Tuesday, November 12, 13
B-trees

®

Tuesday, November 12, 13
B-tree Overview - vocabulary

Pivots
Pointers

Internal Nodes Path to data

Leaf Nodes Actual Data Sorted
®

Tuesday, Nove...
B-tree Overview - example
22

10

2, 3, 4

10,20

99

22,25

99

* Pivot Rule is >=

®

Tuesday, November 12, 13
B-tree Overview - search
“Find 25”
22

10

2, 3, 4

10,20

99

22,25

99

®

Tuesday, November 12, 13
B-tree Overview - insert
“Insert 15”
22

10

2, 3, 4

10,15,20

99

22,25

99

®

Tuesday, November 12, 13
B-tree Overview - performance
Performance is IO limited when data > RAM,
one IO is needed for each insert/update
(actually...
Fractal Tree Indexes

®

Tuesday, November 12, 13
Fractal Tree Indexes
message
buffer

message
buffer

All internal nodes
have message
buffers

message
buffer

As buffers o...
Fractal Tree Indexes - sample data
25

10

2,3,4

10,20

99

22,25

99

Looks a lot like a b-tree!

®

Tuesday, November 1...
Fractal Tree Indexes - insert
insert 15;
insert (15)

25

10

2,3,4

•
•
•
•

99

10,20

22,25

99

search operations must...
Fractal Tree Indexes - other operations
25

delete(8)
delete(2)
insert (8)

2,3,4

10

10,20

add_column(c4 bigint)
delete...
TokuDB
Fractal Tree Indexing +
MySQL/MariaDB

®

Tuesday, November 12, 13
What is TokuDB?

Transactional MySQL Storage Engine - think InnoDB
Available for MySQL 5.5 and MariaDB 5.5
ACID and MVCC
F...
TokuDB Performance
Warning - Benchmarks Ahead!

®

Tuesday, November 12, 13
Indexed Insertion Performance
• High-performance insert/update/delete for large
databases (> RAM) while maintaining indexe...
Sysbench Performance
Sysbench read/write workload, > RAM

The fastest IO is the one you never have to do (compression)
23
...
Performance Advantages
•
•

•
•

Efficient index maintenance, especially secondary
indexes
Clustered secondary indexes
• A...
TokuDB Compression

®

Tuesday, November 12, 13
Compression: TokuDB vs. InnoDB

• InnoDB compression misses force node splits, which
greatly reduces performance
– MySQL 5...
Compression + IO Reduction

• Server was at 90% IO utilization with InnoDB,
10% IO utilization with TokuDB

27

Tuesday, N...
Compression Performance
• iiBench benchmark

28

Tuesday, November 12, 13

®
Compression Achieved
• log data (extremely compressible)

29

Tuesday, November 12, 13

®
TokuDB Agility

®

Tuesday, November 12, 13
The Challenge of MySQL Schema Changes
• Common schema changes can take hours in
MySQL
– Adding, dropping, or expanding a c...
Schema Changes Without Downtime
• In TokuDB, column add/drop/expand is
instantaneous
– “it’s just a message”
• Indexes can...
TokuMX
Fractal Tree Indexing +
MongoDB

®

Tuesday, November 12, 13
What is TokuMX?

• TokuMX = MongoDB with improved storage (Fractal Tree indexes)
• Drop in replacement for MongoDB v2.2 ap...
MongoDB Storage
memory mapped heap

db.test.insert({foo:55})
db.test.ensureIndex({foo:1})

PK index (_id + pointer)

Secon...
TokuMX Storage
db.test.insert({foo:55})
db.test.ensureIndex({foo:1})

memory mapped heap

PK index (_id + document)

Secon...
TokuMX Performance

®

Tuesday, November 12, 13
Performance - Indexed Insertion
• 100mm inserts into a collection with 3 secondary indexes

38

Tuesday, November 12, 13

...
Performance - Inserts on Indexed Arrays
• Indexed Insertion : Multikey (100 inserts per doc)

39

Tuesday, November 12, 13...
Performance - Replication
• TokuMX replication allows secondary servers to process
replication without IO
– Simply injecti...
Performance - Lock Refinement
• TokuMX performs locking at the document level
– Extreme concurrency!
instance
database
col...
Performance - Lock Refinement

42

Tuesday, November 12, 13

®
Performance - Lock Refinement + Reduced IO
• Sysbench benchmark (> RAM)

43

Tuesday, November 12, 13

®
Performance - Reduced IO
– Indexed insertion benchmark

44

Tuesday, November 12, 13

®
Performance - Clustered Indexes
•

Clustered secondary indexes
• Additional copy of the document is stored in the index
• ...
Performance - Memory Management

• Two approaches to memory management
– MongoDB = memory-mapped files
o Operating system ...
TokuMX Compression

®

Tuesday, November 12, 13
Compression
• MongoDB does not offer compression
– Compressed file systems?
– Shortened field names?
o Remember: each fiel...
Compression
• 31 million documents, bit torrent peer data
– http://cs.brown.edu/~pavlo/torrent/

49

Tuesday, November 12,...
TokuMX Transactions

®

Tuesday, November 12, 13
ACID + MVCC
• ACID
– In MongoDB, multi-insertion operations allow for
partial success
o Asked to store 5 documents, 3 succ...
Multi-statement Transactions

• TokuMX brings the following to MongoDB
– db.runCommand({“beginTransaction”, “isolation”:
“...
Questions?

Tim Callaghan
VP/Engineering, Tokutek
tim@tokutek.com
@tmcallaghan

®

Tuesday, November 12, 13
Upcoming SlideShare
Loading in...5
×

Fractal Tree Indexes : From Theory to Practice

1,710

Published on

Fractal Tree Indexes are compared to the indexing incumbent, B-trees. The capabilities are then shown what they bring to MySQL (in TokuDB) and MongoDB (in TokuMX).

Presented at Percona Live London 2013.

Published in: Technology
0 Comments
9 Likes
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total Views
1,710
On Slideshare
0
From Embeds
0
Number of Embeds
3
Actions
Shares
0
Downloads
32
Comments
0
Likes
9
Embeds 0
No embeds

No notes for slide

Fractal Tree Indexes : From Theory to Practice

  1. 1. ® Fractal Tree Indexes Theory to Practice Percona Live London 2013 Tim Callaghan, Tokutek tim@tokutek.com @tmcallaghan ® Tuesday, November 12, 13
  2. 2. Ever seen this? IO Utilization Graph, performance is IO limited ® Tuesday, November 12, 13
  3. 3. Who is Tokutek? Tokutek builds highperformance database software! TokuDB - storage engine for MySQL and MariaDB TokuMX - storage engine for MongoDB Developer Interface" " Storage Engine" HDD & SSD! storage" ® Tuesday, November 12, 13
  4. 4. Who am I? • 17 year database consumer • schema design, development, deployment • database administration + infrastructure • mostly Oracle • 5 year database producer • 2 years @ VoltDB • 2+ years @ Tokutek ® Tuesday, November 12, 13
  5. 5. Housekeeping • Feedback is important to me • Ideas for Webinars or Presentations? • Who’s using MongoDB? • Anyone using TokuDB or TokuMX? • Please ask questions ® Tuesday, November 12, 13
  6. 6. Agenda • Why Fractal Tree indexes are cool • What they enable in MySQL® (TokuDB) • What they enable in MongoDB® (TokuMX) • Q+A ® Tuesday, November 12, 13
  7. 7. Indexing: B-trees and Fractal Tree Indexes ® Tuesday, November 12, 13
  8. 8. B-trees ® Tuesday, November 12, 13
  9. 9. B-tree Overview - vocabulary Pivots Pointers Internal Nodes Path to data Leaf Nodes Actual Data Sorted ® Tuesday, November 12, 13
  10. 10. B-tree Overview - example 22 10 2, 3, 4 10,20 99 22,25 99 * Pivot Rule is >= ® Tuesday, November 12, 13
  11. 11. B-tree Overview - search “Find 25” 22 10 2, 3, 4 10,20 99 22,25 99 ® Tuesday, November 12, 13
  12. 12. B-tree Overview - insert “Insert 15” 22 10 2, 3, 4 10,15,20 99 22,25 99 ® Tuesday, November 12, 13
  13. 13. B-tree Overview - performance Performance is IO limited when data > RAM, one IO is needed for each insert/update (actually it’s one IO for every index on the table) RAM 22 10 99 RAM DISK 2, 3, 4 10,20 22,25 99 ® Tuesday, November 12, 13
  14. 14. Fractal Tree Indexes ® Tuesday, November 12, 13
  15. 15. Fractal Tree Indexes message buffer message buffer All internal nodes have message buffers message buffer As buffers overflow, they cascade down the tree Messages are eventually applied to leaf nodes similar to B-trees •store data in leaf nodes •use index key for ordering different than B-trees •message buffers •big nodes (4MB vs. ~16KB) ® Tuesday, November 12, 13
  16. 16. Fractal Tree Indexes - sample data 25 10 2,3,4 10,20 99 22,25 99 Looks a lot like a b-tree! ® Tuesday, November 12, 13
  17. 17. Fractal Tree Indexes - insert insert 15; insert (15) 25 10 2,3,4 • • • • 99 10,20 22,25 99 search operations must consider messages along the way messages cascade down the tree as buffers fill up they are eventually applied to the leaf nodes, hundreds or thousands of operations for a single IO CPU and cache are conserved as important data is not ejected ® Tuesday, November 12, 13
  18. 18. Fractal Tree Indexes - other operations 25 delete(8) delete(2) insert (8) 2,3,4 10 10,20 add_column(c4 bigint) delete(99) increment(22,+5) ... 99 22,25 insert (100) 99 Lots of operations can be messages! ® Tuesday, November 12, 13
  19. 19. TokuDB Fractal Tree Indexing + MySQL/MariaDB ® Tuesday, November 12, 13
  20. 20. What is TokuDB? Transactional MySQL Storage Engine - think InnoDB Available for MySQL 5.5 and MariaDB 5.5 ACID and MVCC Free/OSS Community Edition – http://github.com/Tokutek/ft-engine • Enterprise Edition – Commercial support + hot backup • • • • Performance + Compression + Agility 20 Tuesday, November 12, 13 ®
  21. 21. TokuDB Performance Warning - Benchmarks Ahead! ® Tuesday, November 12, 13
  22. 22. Indexed Insertion Performance • High-performance insert/update/delete for large databases (> RAM) while maintaining indexes * old numbers, now > 25K/sec 22 Tuesday, November 12, 13 ®
  23. 23. Sysbench Performance Sysbench read/write workload, > RAM The fastest IO is the one you never have to do (compression) 23 Tuesday, November 12, 13 ®
  24. 24. Performance Advantages • • • • Efficient index maintenance, especially secondary indexes Clustered secondary indexes • Additional copy of the row is stored in the index • No additional IO to get row data from primary key • Think better covering index (all non-indexed columns) • Compression eliminates size concerns Big blocks = sequential IO for range scans • Basement nodes are always co-located Multi-threaded bulk loader ® 24 Tuesday, November 12, 13
  25. 25. TokuDB Compression ® Tuesday, November 12, 13
  26. 26. Compression: TokuDB vs. InnoDB • InnoDB compression misses force node splits, which greatly reduces performance – MySQL 5.6 “dynamic padding” (from FB), less cache • Larger block size and flexible on-disk size wins! • Multiple compression algorithms (lzma, quicklz, zlib) • Larger, less frequent writes (much less IO) • Why it matters on spinning disks: – Compressed reads and amortized compressed writes overcome IO limitations • Why it matters on flash/SSD: – Buy less : 250GB * 10x = as 2.5TB) – Large/less frequent writes are flash friendly 26 Tuesday, November 12, 13 ®
  27. 27. Compression + IO Reduction • Server was at 90% IO utilization with InnoDB, 10% IO utilization with TokuDB 27 Tuesday, November 12, 13 ®
  28. 28. Compression Performance • iiBench benchmark 28 Tuesday, November 12, 13 ®
  29. 29. Compression Achieved • log data (extremely compressible) 29 Tuesday, November 12, 13 ®
  30. 30. TokuDB Agility ® Tuesday, November 12, 13
  31. 31. The Challenge of MySQL Schema Changes • Common schema changes can take hours in MySQL – Adding, dropping, or expanding a column – Adding an index • And the table is unavailable for writes during the process • As a workaround, people generally – Use a replication slave, then swap with master – Use helper tools: Percona OSC, MySQL 5.6 o These have IO, CPU, RAM consequences 31 Tuesday, November 12, 13 ®
  32. 32. Schema Changes Without Downtime • In TokuDB, column add/drop/expand is instantaneous – “it’s just a message” • Indexes can be created in the background while table is fully available – TokuDB just builds the index, it does not rebuild the table (MySQL getting better) 32 Tuesday, November 12, 13 ®
  33. 33. TokuMX Fractal Tree Indexing + MongoDB ® Tuesday, November 12, 13
  34. 34. What is TokuMX? • TokuMX = MongoDB with improved storage (Fractal Tree indexes) • Drop in replacement for MongoDB v2.2 applications – Including replication and sharding – Same data model – Same query language – Drivers just work • Open Source – http://github.com/Tokutek/mongo Performance + Compression + Transactions ® Tuesday, November 12, 13
  35. 35. MongoDB Storage memory mapped heap db.test.insert({foo:55}) db.test.ensureIndex({foo:1}) PK index (_id + pointer) Secondary index (foo + pointer) 18 85 4 (1,ptr5) (4,ptr1), (12,ptr8) 5555 (19,ptr7) 40 (10000,ptr2) (2,ptr5), (22,ptr6) (50,ptr4) 120 (100,ptr7) (222,ptr3) The “pointer” tells MongoDB where to look in the heap for the requested document (another IO) ® 35 Tuesday, November 12, 13
  36. 36. TokuMX Storage db.test.insert({foo:55}) db.test.ensureIndex({foo:1}) memory mapped heap PK index (_id + document) Secondary index (foo + _id) 18 4 (1,doc) (4,doc), (12,doc) 85 5555 (19,doc) 40 (10000,doc) (2,4), (22,12) (50,19) 120 (100,10000) (222,1) One less IO per _id lookup, document is clustered in the index ® 36 Tuesday, November 12, 13
  37. 37. TokuMX Performance ® Tuesday, November 12, 13
  38. 38. Performance - Indexed Insertion • 100mm inserts into a collection with 3 secondary indexes 38 Tuesday, November 12, 13 ®
  39. 39. Performance - Inserts on Indexed Arrays • Indexed Insertion : Multikey (100 inserts per doc) 39 Tuesday, November 12, 13 ®
  40. 40. Performance - Replication • TokuMX replication allows secondary servers to process replication without IO – Simply injecting messages into the Fractal Tree Indexes on the secondary server – The “Hard Work” was done on the primary o Uniqueness checking o Transactional locking o Update effort (read-before-write) – Elimination of replication lag • Your secondaries are fully available for read scaling! – Wasn’t that the point? 40 Tuesday, November 12, 13 ®
  41. 41. Performance - Lock Refinement • TokuMX performs locking at the document level – Extreme concurrency! instance database collection MongoDB v2.0 database collection collection collection document document document document document document document MongoDB v2.2 TokuMX document document document 41 Tuesday, November 12, 13 ®
  42. 42. Performance - Lock Refinement 42 Tuesday, November 12, 13 ®
  43. 43. Performance - Lock Refinement + Reduced IO • Sysbench benchmark (> RAM) 43 Tuesday, November 12, 13 ®
  44. 44. Performance - Reduced IO – Indexed insertion benchmark 44 Tuesday, November 12, 13 ®
  45. 45. Performance - Clustered Indexes • Clustered secondary indexes • Additional copy of the document is stored in the index • No additional IO to get row data from primary key • Think better covered index (all non-indexed fields) • Good for point queries, great for range scans • Compression eliminates size concerns 45 Tuesday, November 12, 13 ®
  46. 46. Performance - Memory Management • Two approaches to memory management – MongoDB = memory-mapped files o Operating system determines what data is important – TokuMX = managed cache o User defined size o TokuMX determines what data is important • Run multiple TokuMX instances on a single server – Each has it’s own fixed cache size 46 Tuesday, November 12, 13 ®
  47. 47. TokuMX Compression ® Tuesday, November 12, 13
  48. 48. Compression • MongoDB does not offer compression – Compressed file systems? – Shortened field names? o Remember: each field name is stored in every single document • TokuMX easily achieves 5x-10x compression – Buy less disk or flash – Compressed reads and writes reduce overall IO • TokuMX support 3 compression types – zlib, quicklz, lzma (size vs. speed) – all data is compressed • Use descriptive field names! – They are easy to compress 48 Tuesday, November 12, 13 ®
  49. 49. Compression • 31 million documents, bit torrent peer data – http://cs.brown.edu/~pavlo/torrent/ 49 Tuesday, November 12, 13 ®
  50. 50. TokuMX Transactions ® Tuesday, November 12, 13
  51. 51. ACID + MVCC • ACID – In MongoDB, multi-insertion operations allow for partial success o Asked to store 5 documents, 3 succeeded – We offer “all or nothing” behavior – Document level locking • MVCC – In MongoDB, queries can be interrupted by writers. o The effect of these writers are visible to the reader – TokuMX offers MVCC o Reads are consistent as of the operation start 51 Tuesday, November 12, 13 ®
  52. 52. Multi-statement Transactions • TokuMX brings the following to MongoDB – db.runCommand({“beginTransaction”, “isolation”: “mvcc”}) – ... perform 1 or more operations – db.runCommand(“rollbackTransaction”) | db.runCommand(“commitTransaction”) • Not allowed in sharded environments – mongos will reject 52 Tuesday, November 12, 13 ®
  53. 53. Questions? Tim Callaghan VP/Engineering, Tokutek tim@tokutek.com @tmcallaghan ® Tuesday, November 12, 13
  1. A particular slide catching your eye?

    Clipping is a handy way to collect important slides you want to go back to later.

×