Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

分散ストレージに使えるかもしれないアルゴリズム

5,928 views

Published on

Published in: Technology
  • 経路制御 Chord EpiChord
    トランザクション VectorClocks Sinfonia
    集合操作 Bloomフィルタ BDD ZDD
       Reply 
    Are you sure you want to  Yes  No
    Your message goes here
  • 経路制御 Chord EpiChord 管理ノード数とホップ数のトレードオフのグラフ
    トランザクション VectorClocks 2-phase commitの最適化(Sinfonia)
    集合操作 BDD ZDD
       Reply 
    Are you sure you want to  Yes  No
    Your message goes here

分散ストレージに使えるかもしれないアルゴリズム

  1. 1. 2010-02-26 4 Erlang
  2. 2. 1/2   No Erlang       3     Chord   VectorClocks   …
  3. 3. 2/2         Chord EpiChord BASE Sinfonia: VectorClocks Bloom ZDD EpiChord
  4. 4. Lookup     1 0-10 1 5? 11-20 2 2 21-30 3 3           !   ?
  5. 5. B. Leong et al. http://www.comp.nus.edu.sg/~bleong/slides/icon-epichord-slides.pdf Address Space 0-63
  6. 6. B. Leong et al. http://www.comp.nus.edu.sg/~bleong/slides/icon-epichord-slides.pdf Mapping Keys to Nodes 1-6 6 : : 52-57 57 : :
  7. 7. B. Leong et al. http://www.comp.nus.edu.sg/~bleong/slides/icon-epichord-slides.pdf Chord 35 36-40 40 41-47 47 48-50 49 51-2 57 3-30 6 •  log •  log
  8. 8. B. Leong et al. http://www.comp.nus.edu.sg/~bleong/slides/icon-epichord-slides.pdf EpiChord Lookup Algorithm
  9. 9. B. Leong et al. http://www.comp.nus.edu.sg/~bleong/slides/icon-epichord-slides.pdf EpiChord Lookup Algorithm
  10. 10. B. Leong et al. http://www.comp.nus.edu.sg/~bleong/slides/icon-epichord-slides.pdf EpiChord Lookup Algorithm
  11. 11. B. Leong et al. http://www.comp.nus.edu.sg/~bleong/slides/icon-epichord-slides.pdf EpiChord Lookup Algorithm
  12. 12. B. Leong et al. http://www.comp.nus.edu.sg/~bleong/slides/icon-epichord-slides.pdf EpiChord Lookup Algorithm
  13. 13. B. Leong et al. http://www.comp.nus.edu.sg/~bleong/slides/icon-epichord-slides.pdf EpiChord Lookup Algorithm
  14. 14. B. Leong et al. http://www.comp.nus.edu.sg/~bleong/slides/icon-epichord-slides.pdf EpiChord Lookup Algorithm
  15. 15. B. Leong et al. http://www.comp.nus.edu.sg/~bleong/slides/icon-epichord-slides.pdf EpiChord Lookup Algorithm
  16. 16. B. Leong et al. http://www.comp.nus.edu.sg/~bleong/slides/icon-epichord-slides.pdf EpiChord Division of Address Space ?
  17. 17. J. Xu et al., On the Fundamental Tradeoffs between Routing Table Size and Network Diameter in Peer-to-Peer Networks Lookup (Dynamo?) EpiChord?
  18. 18.       ACID: 1.  SELECT … 2.  INSERT … 3.  : DB     1 ?     !   ?
  19. 19. S. Shinohara http://www.slideshare.net/shino/conflict-resolution-in-kai-presentation VectorClocks “happens-before” (->) BASE VectorClocks
  20. 20. S. Shinohara http://www.slideshare.net/shino/conflict-resolution-in-kai-presentation VectorClocks “concurrent” (||) BASE VectorClocks
  21. 21. T.Yamamuro (postgresql’s) 2-phase commit
  22. 22. T.Yamamuro Sinfonia 2-phase commit’s optimization ?
  23. 23. T.Yamamuro Sinfonia mini-transaction primitives
  24. 24. T.Yamamuro Sinfonia Sinfonia’s 2-phase commit =
  25. 25. T.Yamamuro Sinfonia Sinfonia 2-phase commit 5
  26. 26. T.Yamamuro Sinfonia Sinfonia 2-phase commit 5
  27. 27. T.Yamamuro Sinfonia Sinfonia 2-phase commit 5
  28. 28. T.Yamamuro Sinfonia Evaluation: Scalability NFS, 250 2 100
  29. 29. VectorClocks Sinfonia 2-phase commit BASE ACID
  30. 30.     {apple, banana} + {orange} Is apple a member? = {apple, banana, orange} {apple, banana, orange}           !   ? Q. ? A. BigTable Bloom
  31. 31. Bloom filter   BF m 0 0 0 0 0 0 0 0 ZF m hi i 0<i k [0,m-1] hi(x)   0 1 0 0 1 0 0 0   BF = ZF     BF[ hi(x) ] = 1 hi(y) 0 1 0 0 1 1 0 0 Bloom filter m=8, k=2 x, y
  32. 32. Bloom filter   membership query   F = ZF hi(x) included   F[ hi(x) ] = 1   BF & F == F 0 1 0 0 1 1 0 0 hi(z) not included 0 1 0 0 1 1 0 0 false positive! hi(w) included 0 1 0 0 1 1 0 0
  33. 33. Bloom filter     O k   n  
  34. 34. Bloom filter   OR AND   m, hi 0 1 0 0 1 1 0 0 0 0 0 1 1 0 0 1 0 1 0 1 1 1 0 1
  35. 35. Bloom filter false positive   p  1  kn  kn  p = 1−  ≈ exp −   m  m   f € k f = (1− p) € m = 256, n = 32
  36. 36. Bloom filter false positive   f k   f k  m k = ln2 ⋅   n   floor   k   €   f m f ≈ 0.6185 n k
  37. 37. Bloom filter   n, f m m = −1.44n log 2 f k €
  38. 38. Bloom filter   b b = −1.44 log 2 f k   b = −log 2 f 9.6 1% €   1.44   € 9.6   Bloom filter   b 1%
  39. 39. Bloom filter   k p 1 p= 0 1 0 0 1 1 0 1 2 k Bloom filter   [mitzenmacher02] €   k   0 1  
  40. 40. S. Minato http://www-alg.ist.hokudai.ac.jp/~minato/alg2009-j.html ZDD BDD (Binary Decision Diagram) {abc, ab, ac, b, } trie 0 (false) 1 (true)
  41. 41. S. Minato http://www-alg.ist.hokudai.ac.jp/~minato/alg2009-j.html ZDD BDD
  42. 42. S. Minato http://www-alg.ist.hokudai.ac.jp/~minato/alg2009-j.html ZDD BDD
  43. 43. S. Minato http://www-alg.ist.hokudai.ac.jp/~minato/alg2009-j.html ZDD
  44. 44. S. Minato http://www-alg.ist.hokudai.ac.jp/~minato/alg2009-j.html ZDD
  45. 45. S. Minato http://www-alg.ist.hokudai.ac.jp/~minato/alg2009-j.html ZDD BDD ZDD: Zero-suppressed BDD
  46. 46. S. Minato http://www-alg.ist.hokudai.ac.jp/~minato/alg2009-j.html ZDD ZDD
  47. 47. S. Minato http://www-alg.ist.hokudai.ac.jp/~minato/alg2009-j.html ZDD ZDD
  48. 48. S. Minato http://www-alg.ist.hokudai.ac.jp/~minato/alg2009-j.html ZDD ZDD
  49. 49.         EpiChord   B. Leong et al., “EpiChord: Parallelizing the Chord lookup algorithm with reactive routing state management,” Comput. Commun., vol.29, no.9, pp.1243-1259, 2006.     Sinfonia   M.K. Aguilera et al., “Sinfonia: a new paradigm for building scalable distributed systems,” Proc. of SOSP’07, pp.159-174, 2007.     ZDD   S. Minato, “Zero-suppressed BDDs for set manipulation in combinatorial problems,” Proc. of DAC’93, pp. 272-277, 1993.  

×