Константин Осипов, Mail.Ru, Tarantool

2,300 views

Published on

HighLoad++ 2013

Published in: Technology, Business
0 Comments
2 Likes
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total views
2,300
On SlideShare
0
From Embeds
0
Number of Embeds
1,015
Actions
Shares
0
Downloads
30
Comments
0
Likes
2
Embeds 0
No embeds

No notes for slide

Константин Осипов, Mail.Ru, Tarantool

  1. 1. Популярные алгоритмы хранения данных на диске Konstantin Osipov, kostja@tarantool.org October 28th, 2013
  2. 2. Случай в квадрате 36-80 • • • B-tree – most popular disk-based data structure B-tree balances INSERT, UPDATE and SELECT speed DELETEs can be slow
  3. 3. СУБД быстрая, настраивать надо уметь
  4. 4. B-tree: внутреннее устройство
  5. 5. Что означает сache-oblivious?
  6. 6. Что означает сache-oblivious? (2) BLOCK­MULT(A,B,C,n): 1 for i = 1 to n/s do: 2    for j = 1 to n/s do: 3         for k = 1 to n/s do: 4             ORD­MULT(Aik, Bkj, Cij, s)
  7. 7. LSM-tree: внутреннее устройство
  8. 8. LSM-tree: внутреннее устройство (2)
  9. 9. LevelDB: устройство
  10. 10. LevelDB: insert RPS
  11. 11. LSM-tree: применение ● Данные с разной степенью актуальности – – Стена в соцсети – Чаты – ● Ленты сообщений События Сегрегация данных – Данные в LSM space, индекс в MEMORY space
  12. 12. TokuDB/CO lookahead arrays
  13. 13. PUT(37), PUT(16) Self-Balancing Tree Memory Disk WAL:
  14. 14. 16 37 Self-Balancing Tree Memory Disk WAL: 37, 16
  15. 15. 7 41 Self-Balancing Tree Memory 16 37 Sorted String Table WAL: 41, 7, 37, 16 Disk
  16. 16. Memory 7 37 7 16 37 41 WAL: 41, 7, 28, 16 Disk
  17. 17. 10 28 Memory 7 37 Disk 7 16 37 41 WAL: 10, 28, 41, 7, 37, 16
  18. 18. Memory 10 28 7 16 37 41 WAL: 10, 28, 41, 7, 37, 16 Disk
  19. 19. 2 47 Memory 10 28 Disk 7 16 37 41 WAL: 47, 2, 10, 28, 41, 7, 37, 16
  20. 20. 2 28 2 10 28 41 2 7 10 16 28 37 41 47 WAL: 47, 2, 10, 28, 41, 7, 37, 16 Memory Disk
  21. 21. 6 49 2 28 2 10 28 41 2 7 10 16 28 37 41 47 WAL: 49, 6, 47, 2, 10, 28, 41, 7, 37, 16 Memory Disk
  22. 22. 6 49 2 10 28 41 2 7 10 16 28 37 41 47 WAL: 49, 6, 47, 2, 10, 28, 41, 7, 37, 16 Memory Disk
  23. 23. 23 32 6 49 2 10 28 41 2 7 10 16 28 37 41 47 WAL: 32, 23, 49, 6, 47, 2, 10, 28, 41, 7, 37, 16 Memory Disk
  24. 24. 6 32 6 23 32 49 2 7 10 16 28 37 41 47 WAL: 32, 23, 49, 6, 47, 2, 10, 28, 41, 7, 37, 16 Memory Disk
  25. 25. 30 45 Memory 6 32 Disk 6 23 32 49 2 7 10 16 28 37 41 47 WAL: 30, 45, 32, 23, 49, 6, 47, 2, 10, 28, 41, 7, 37, 16
  26. 26. 14 38 Memory 30 45 Disk 6 23 32 49 2 7 10 16 28 37 41 47 WAL: 38, 14, 30, 45, 32, 23, 49, 6, 47, 2, 10, 28, 41, 7, 37, 16
  27. 27. 6 10 Memory 2 30 Disk 2 14 30 41 2 2 6 7 14 23 30 37 41 47 7 10 14 16 23 28 30 32 37 38 41 45 47 49 WAL: 10, 6, 38, 14, 45, 30, 45, 32, 23, 49, 6, 47, 2, 10, 28, 41, 7, 37, 16
  28. 28. Memory 22 37 Disk 10 25 36 42 3 2 6 8 15 26 35 40 45 48 7 10 14 16 23 28 30 32 37 38 41 45 47 49 WAL: 37, 22, 36, 10, 25, 42, 10, 6, 38, 14, 45, 30, 45, 32, 23, 49, 6, 47, 2, 10, 28, 41, 7, 37, 16
  29. 29. GET(16) Memory 22 37 Disk 10 25 36 42 3 2 6 8 15 26 35 40 45 48 7 10 14 16 23 28 30 32 37 38 41 45 47 49 WAL: 37, 22, 36, 10, 25, 42, 10, 6, 38, 14, 45, 30, 45, 32, 23, 49, 6, 47, 2, 10, 28, 41, 7, 37, 16
  30. 30. GET(16) Memory 22 37 Disk 10 25 36 42 3 2 6 8 15 26 35 40 45 48 7 10 14 16 23 28 30 32 37 38 41 45 47 49 WAL: 37, 22, 36, 10, 25, 42, 10, 6, 38, 14, 45, 30, 45, 32, 23, 49, 6, 47, 2, 10, 28, 41, 7, 37, 16
  31. 31. BitCask: AOF format
  32. 32. BitCask: key dir
  33. 33. Sophia:
  34. 34. Links ● ● ● ● ● ● ● ● ● ● Bitcask A Log-Structured Hash Table for Fast Key/Value Data, Justin Sheehy David Smith with inspiration from Eric Brewer The Log-Structured Merge-Tree (LSM-Tree) Patrick O'Neil , Edward Cheng, Dieter Gawlick, Elizabeth O'Neil Cache-Oblivious Algorithms by Harald Prokop (Master theses) Space/time trade-offs in hash coding with allowable errors, Burton H. Bloom Data Structures and Algorithms for Big Databases, Michael A. Bender Stony Brook & Tokutek Bradley C. Kuszmaul (XLDB tutorial) http://github.com/pmwkaa/sophia, http://sphia.org http://codecapsule.com/2012/12/30/implementing-a-key-value-store-part-3-comparative-analysis-of-the-ar http://stackoverflow.com/questions/6079890/cache-oblivious-lookahead-array http://www.youtube.com/watch?v=88NaRUdoWZM(Tim Callaghan: Fractal Tree indexes) http://code.google.com/p/leveldb/downloads/list
  35. 35. Эпилог: choose your db wisely ?

×