分散ストレージに使えるかもしれないアルゴリズム

5,530 views
5,366 views

Published on

Published in: Technology
2 Comments
20 Likes
Statistics
Notes
  • 経路制御 Chord EpiChord
    トランザクション VectorClocks Sinfonia
    集合操作 Bloomフィルタ BDD ZDD
       Reply 
    Are you sure you want to  Yes  No
    Your message goes here
  • 経路制御 Chord EpiChord 管理ノード数とホップ数のトレードオフのグラフ
    トランザクション VectorClocks 2-phase commitの最適化(Sinfonia)
    集合操作 BDD ZDD
       Reply 
    Are you sure you want to  Yes  No
    Your message goes here
No Downloads
Views
Total views
5,530
On SlideShare
0
From Embeds
0
Number of Embeds
1,131
Actions
Shares
0
Downloads
0
Comments
2
Likes
20
Embeds 0
No embeds

No notes for slide

分散ストレージに使えるかもしれないアルゴリズム

  1. 1. 2010-02-26 4 Erlang
  2. 2. 1/2   No Erlang       3     Chord   VectorClocks   …
  3. 3. 2/2         Chord EpiChord BASE Sinfonia: VectorClocks Bloom ZDD EpiChord
  4. 4. Lookup     1 0-10 1 5? 11-20 2 2 21-30 3 3           !   ?
  5. 5. B. Leong et al. http://www.comp.nus.edu.sg/~bleong/slides/icon-epichord-slides.pdf Address Space 0-63
  6. 6. B. Leong et al. http://www.comp.nus.edu.sg/~bleong/slides/icon-epichord-slides.pdf Mapping Keys to Nodes 1-6 6 : : 52-57 57 : :
  7. 7. B. Leong et al. http://www.comp.nus.edu.sg/~bleong/slides/icon-epichord-slides.pdf Chord 35 36-40 40 41-47 47 48-50 49 51-2 57 3-30 6 •  log •  log
  8. 8. B. Leong et al. http://www.comp.nus.edu.sg/~bleong/slides/icon-epichord-slides.pdf EpiChord Lookup Algorithm
  9. 9. B. Leong et al. http://www.comp.nus.edu.sg/~bleong/slides/icon-epichord-slides.pdf EpiChord Lookup Algorithm
  10. 10. B. Leong et al. http://www.comp.nus.edu.sg/~bleong/slides/icon-epichord-slides.pdf EpiChord Lookup Algorithm
  11. 11. B. Leong et al. http://www.comp.nus.edu.sg/~bleong/slides/icon-epichord-slides.pdf EpiChord Lookup Algorithm
  12. 12. B. Leong et al. http://www.comp.nus.edu.sg/~bleong/slides/icon-epichord-slides.pdf EpiChord Lookup Algorithm
  13. 13. B. Leong et al. http://www.comp.nus.edu.sg/~bleong/slides/icon-epichord-slides.pdf EpiChord Lookup Algorithm
  14. 14. B. Leong et al. http://www.comp.nus.edu.sg/~bleong/slides/icon-epichord-slides.pdf EpiChord Lookup Algorithm
  15. 15. B. Leong et al. http://www.comp.nus.edu.sg/~bleong/slides/icon-epichord-slides.pdf EpiChord Lookup Algorithm
  16. 16. B. Leong et al. http://www.comp.nus.edu.sg/~bleong/slides/icon-epichord-slides.pdf EpiChord Division of Address Space ?
  17. 17. J. Xu et al., On the Fundamental Tradeoffs between Routing Table Size and Network Diameter in Peer-to-Peer Networks Lookup (Dynamo?) EpiChord?
  18. 18.       ACID: 1.  SELECT … 2.  INSERT … 3.  : DB     1 ?     !   ?
  19. 19. S. Shinohara http://www.slideshare.net/shino/conflict-resolution-in-kai-presentation VectorClocks “happens-before” (->) BASE VectorClocks
  20. 20. S. Shinohara http://www.slideshare.net/shino/conflict-resolution-in-kai-presentation VectorClocks “concurrent” (||) BASE VectorClocks
  21. 21. T.Yamamuro (postgresql’s) 2-phase commit
  22. 22. T.Yamamuro Sinfonia 2-phase commit’s optimization ?
  23. 23. T.Yamamuro Sinfonia mini-transaction primitives
  24. 24. T.Yamamuro Sinfonia Sinfonia’s 2-phase commit =
  25. 25. T.Yamamuro Sinfonia Sinfonia 2-phase commit 5
  26. 26. T.Yamamuro Sinfonia Sinfonia 2-phase commit 5
  27. 27. T.Yamamuro Sinfonia Sinfonia 2-phase commit 5
  28. 28. T.Yamamuro Sinfonia Evaluation: Scalability NFS, 250 2 100
  29. 29. VectorClocks Sinfonia 2-phase commit BASE ACID
  30. 30.     {apple, banana} + {orange} Is apple a member? = {apple, banana, orange} {apple, banana, orange}           !   ? Q. ? A. BigTable Bloom
  31. 31. Bloom filter   BF m 0 0 0 0 0 0 0 0 ZF m hi i 0<i k [0,m-1] hi(x)   0 1 0 0 1 0 0 0   BF = ZF     BF[ hi(x) ] = 1 hi(y) 0 1 0 0 1 1 0 0 Bloom filter m=8, k=2 x, y
  32. 32. Bloom filter   membership query   F = ZF hi(x) included   F[ hi(x) ] = 1   BF & F == F 0 1 0 0 1 1 0 0 hi(z) not included 0 1 0 0 1 1 0 0 false positive! hi(w) included 0 1 0 0 1 1 0 0
  33. 33. Bloom filter     O k   n  
  34. 34. Bloom filter   OR AND   m, hi 0 1 0 0 1 1 0 0 0 0 0 1 1 0 0 1 0 1 0 1 1 1 0 1
  35. 35. Bloom filter false positive   p  1  kn  kn  p = 1−  ≈ exp −   m  m   f € k f = (1− p) € m = 256, n = 32
  36. 36. Bloom filter false positive   f k   f k  m k = ln2 ⋅   n   floor   k   €   f m f ≈ 0.6185 n k
  37. 37. Bloom filter   n, f m m = −1.44n log 2 f k €
  38. 38. Bloom filter   b b = −1.44 log 2 f k   b = −log 2 f 9.6 1% €   1.44   € 9.6   Bloom filter   b 1%
  39. 39. Bloom filter   k p 1 p= 0 1 0 0 1 1 0 1 2 k Bloom filter   [mitzenmacher02] €   k   0 1  
  40. 40. S. Minato http://www-alg.ist.hokudai.ac.jp/~minato/alg2009-j.html ZDD BDD (Binary Decision Diagram) {abc, ab, ac, b, } trie 0 (false) 1 (true)
  41. 41. S. Minato http://www-alg.ist.hokudai.ac.jp/~minato/alg2009-j.html ZDD BDD
  42. 42. S. Minato http://www-alg.ist.hokudai.ac.jp/~minato/alg2009-j.html ZDD BDD
  43. 43. S. Minato http://www-alg.ist.hokudai.ac.jp/~minato/alg2009-j.html ZDD
  44. 44. S. Minato http://www-alg.ist.hokudai.ac.jp/~minato/alg2009-j.html ZDD
  45. 45. S. Minato http://www-alg.ist.hokudai.ac.jp/~minato/alg2009-j.html ZDD BDD ZDD: Zero-suppressed BDD
  46. 46. S. Minato http://www-alg.ist.hokudai.ac.jp/~minato/alg2009-j.html ZDD ZDD
  47. 47. S. Minato http://www-alg.ist.hokudai.ac.jp/~minato/alg2009-j.html ZDD ZDD
  48. 48. S. Minato http://www-alg.ist.hokudai.ac.jp/~minato/alg2009-j.html ZDD ZDD
  49. 49.         EpiChord   B. Leong et al., “EpiChord: Parallelizing the Chord lookup algorithm with reactive routing state management,” Comput. Commun., vol.29, no.9, pp.1243-1259, 2006.     Sinfonia   M.K. Aguilera et al., “Sinfonia: a new paradigm for building scalable distributed systems,” Proc. of SOSP’07, pp.159-174, 2007.     ZDD   S. Minato, “Zero-suppressed BDDs for set manipulation in combinatorial problems,” Proc. of DAC’93, pp. 272-277, 1993.  

×