Rakuten LeoFs - distributed file system

13,097 views

Published on

LeoFS has High Performance, High Reliability, High Scalability, file system developed by Rakuten.

Published in: Technology, Business
0 Comments
18 Likes
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total views
13,097
On SlideShare
0
From Embeds
0
Number of Embeds
3,154
Actions
Shares
0
Downloads
80
Comments
0
Likes
18
Embeds 0
No embeds

No notes for slide

Rakuten LeoFs - distributed file system

  1. S3 Compatible Storage “LeoFS” Rakuten. Inc, RIT Yosuke Hara 25/05/2012 1
  2. Table of Contents 1. Motivation 2. Overview 3. Inside of LeoFS 4. WEB Console 2
  3. 1. Motivation 3
  4. Why NFS? Which is suitable storage for storing the media files? ?Low ROIPossibility of SPOFStorage Expansion is difficult during increasing data 4
  5. 5
  6. Object Storage Farm for IaaS 6
  7. 2. Overview 7
  8. Overview Storage Gateway Manager 8
  9. System Layout Request from Web Application(s) Load Balancer LeoFS-Manager S3-API REST over HTTPLeoFS-Gateway RPCw/Cache Server SNMP RPCLeoFS-Storage Storage Engine/Router Storage Engine/Router Storage Engine/Router WEB Console META Object Store META Object Store META Object Store 9
  10. System Layout Request from Web Application(s) Load Balancer Gateway Manager LeoFS-Manager HTTP Cluster Request/Response Handling S3-API Management REST over HTTPLeoFS-Gateway + RPC w/Object Cachew/Cache Server Ring Watcher (AWS S3-API) Node Watcher SNMP RPCLeoFS-Storage Storage GUI Console Object Storage, Meta data Storage Storage Engine/Router Storage Engine/Router Storage Engine/Router + META Replicator/Recoverer Object Store META META Object Store Object Store 10
  11. 3. Inside of LeoFS 11
  12. Architecture HTTP Gateway (stateless) Erlang RPC Erlang RPC Storage Cluster (multi master) Erlang RPC Process Monitor Manager Cluster 12
  13. ArchitectureArchitecture - Gateway/Storage LeoFS Gateway Cacher REST over HTTP (S3-API) get put delete head redundant-manager membership (fault-detection) RPC LeoFS-Storage redundant-manager replicator read-repairerRPC RPC membership (fault-detection) queue Storage Engine Object Storage Metadata Storage 13
  14. ArchitectureArchitecture - Manager Cluster Erlang Mnesia RING Member / Cluster State Auth / ACL Process Monitor Gateway / Storage Cluster 14
  15. Architecture 15
  16. Three “HIGH” High Cost Performance Monolithic Storage System Storage Engine For Unstructured Files Traffic Restrain Mechanism > File Cache System (Gateway Plugin) 16
  17. Three “HIGH” High Reliability NO SPOF Split Brain Measure “Erlang OTP” > Nine Nines (99.9999999%) 17
  18. Three “HIGH” High Scalability Elastic Storage System > Able to dynamic attach/detach nodes > Able to over 100-nodes > NOT Mesh-connected Mutual Servers 18
  19. Cache Mechanism 19
  20. Cache MechanismGateway Buffer Pool Slab Alloc Skiplist {$filename, $etag} request from Client response Gateway match: {ok, match} NOT match: {ok, $metadata, $body,} High I/O efficiency Low Latency 20
  21. Storage Engine 21
  22. Storage EngineMetadata + Object Storage LeoFS-Storage Storage Engine / Replicator / Recoverer Object’s Attribute Storage (metadata) Object Storage Metadata : Keeps an in-memory index of all data. Object Storage : Log structured (append-only) object store. 22
  23. Storage EngineRetrieve an object from Storage Storage Engine < META DATA > ID Filename Offset Size Checksum Data Header Metadata File Footer Object Container 23
  24. Storage EngineInsert an object into Storage Storage Engine Add a Metadata Meta Data Server Data Append a File 24
  25. Storage EngineReduce unnecessary files Compaction 25
  26. 4. WEB Console 26
  27. Web Console File Manager WEB Console Node Stats Log Search 27
  28. Web Console Node State MonitorLog Analyzer / Searcher 28
  29. Web ConsoleWeb Console System Layout Manager Storage Gateway or GUI Console Producers and Admins Logstatsh 29
  30. Wrap Up 30
  31. Wrap Up High Cost Performance High Reliability High Scalability 31
  32. 32
  33. Thank you for your time 33

×