Loading…

Flash Player 9 (or above) is needed to view presentations.
We have detected that you do not have it on your computer. To install it, go here.

Like this presentation? Why not share!

Евгений Поляков – Распределенные системы хранения данных, особенности реализации DHT в проекте Elliptics network

on

  • 1,504 views

 

Statistics

Views

Total Views
1,504
Views on SlideShare
1,441
Embed Views
63

Actions

Likes
0
Downloads
3
Comments
0

3 Embeds 63

http://webcrunch.ru 60
http://developers.rambler.ru 2
http://bizspark.webcrunch.ru 1

Accessibility

Categories

Upload Details

Uploaded via as Adobe PDF

Usage Rights

CC Attribution-NonCommercial-ShareAlike LicenseCC Attribution-NonCommercial-ShareAlike LicenseCC Attribution-NonCommercial-ShareAlike License

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Processing…
Post Comment
Edit your comment

Евгений Поляков – Распределенные системы хранения данных, особенности реализации DHT в проекте Elliptics network Presentation Transcript

  • 1. Elliptics Network Evgeniy Polyakov <zbr@ioremap.net> <zbr@yandex-team.ru> Distributed hash table Key/value storage
  • 2. How to handle huge dataset? Can existing solutions scale?
  • 3. Existing solutions
  • 4. Distributed hash table Consistent hashing Map and routing table
  • 5. Elliptics network architecture Frontend Core Backend
  • 6. Frontends
  • 7. Frontends: HTTP
  • 8. Frontends: bindings
  • 9. Frontends: command line
  • 10. Frontends: POHMELFS
  • 11. IO backends
  • 12. Eblob random read performance: SAS ● 2 sas shelves (14 disks raid10 each, ext4) ● 1 Tb of data ● ~ 100 millions of objects ● Eblob: 5000 rps ● Eblob: 3500 rps within 100 ms ● Eblob: 4000 rps witin 200 ms ● Filesystem: 600 rps within 200 ms ● Filesystem: 800 rps within 300 ms FS contains about 30 millions of objects actually
  • 13. Eblob random read performance: SATA ● 2 sata raids (4-disks raid10 each, ext4) ● 370 Gb of data ● 30 millions of objects ● Eblob: 1000 rps ● Eblob: 900 rps within 100-150 ms ● Filesystem: 200 rps within 200 ms
  • 14. Elliptics network: core
  • 15. IO models Write always succeed Multiple copy reading Eventual consistency
  • 16. Future plans