Elliptics Network
Evgeniy Polyakov
<zbr@ioremap.net>
<zbr@yandex-team.ru>
Distributed hash table
Key/value storage
How to handle huge dataset?
Can existing solutions scale?
Existing solutions
Distributed hash table
Consistent hashing
Map and routing table
Elliptics network architecture
Frontend
Core
Backend
Frontends
Frontends: HTTP
Frontends: bindings
Frontends: command line
Frontends: POHMELFS
IO backends
Eblob random read performance: SAS
● 2 sas shelves (14 disks raid10 each, ext4)
● 1 Tb of data
● ~ 100 millions of objects...
Eblob random read performance: SATA
● 2 sata raids (4-disks raid10 each, ext4)
● 370 Gb of data
● 30 millions of objects
●...
Elliptics network: core
IO models
Write always succeed
Multiple copy reading
Eventual consistency
Future plans
Евгений Поляков – Распределенные системы хранения данных, особенности реализации DHT в проекте Elliptics network
Upcoming SlideShare
Loading in...5
×

Евгений Поляков – Распределенные системы хранения данных, особенности реализации DHT в проекте Elliptics network

1,250

Published on

0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total Views
1,250
On Slideshare
0
From Embeds
0
Number of Embeds
1
Actions
Shares
0
Downloads
6
Comments
0
Likes
0
Embeds 0
No embeds

No notes for slide

Евгений Поляков – Распределенные системы хранения данных, особенности реализации DHT в проекте Elliptics network

  1. 1. Elliptics Network Evgeniy Polyakov <zbr@ioremap.net> <zbr@yandex-team.ru> Distributed hash table Key/value storage
  2. 2. How to handle huge dataset? Can existing solutions scale?
  3. 3. Existing solutions
  4. 4. Distributed hash table Consistent hashing Map and routing table
  5. 5. Elliptics network architecture Frontend Core Backend
  6. 6. Frontends
  7. 7. Frontends: HTTP
  8. 8. Frontends: bindings
  9. 9. Frontends: command line
  10. 10. Frontends: POHMELFS
  11. 11. IO backends
  12. 12. Eblob random read performance: SAS ● 2 sas shelves (14 disks raid10 each, ext4) ● 1 Tb of data ● ~ 100 millions of objects ● Eblob: 5000 rps ● Eblob: 3500 rps within 100 ms ● Eblob: 4000 rps witin 200 ms ● Filesystem: 600 rps within 200 ms ● Filesystem: 800 rps within 300 ms FS contains about 30 millions of objects actually
  13. 13. Eblob random read performance: SATA ● 2 sata raids (4-disks raid10 each, ext4) ● 370 Gb of data ● 30 millions of objects ● Eblob: 1000 rps ● Eblob: 900 rps within 100-150 ms ● Filesystem: 200 rps within 200 ms
  14. 14. Elliptics network: core
  15. 15. IO models Write always succeed Multiple copy reading Eventual consistency
  16. 16. Future plans
  1. A particular slide catching your eye?

    Clipping is a handy way to collect important slides you want to go back to later.

×