Download to read offline
![Beolink.org
Block Storage Devices
3
Pisa
is a simple block data
distribution and
Replication Framework on a
wide range of node
New Node
Transfer
New Node
New Node
Node
Data
Block
Key
Data
[Hash]](https://image.slidesharecdn.com/pisa2-140527013013-phpapp01/85/Pisa-3-320.jpg)
![Beolink.orgData Distribution
Zero Hop Hash (Consistent HASH)
- Partition location with 0 hops
- 1% capacity added and 1% moved
Node
• Zone
• Weight
Partition , array list (FIXED) :
• Position = kex prefix
• Value = node id
Shuffle
Avoid sequential allocation
Part_list = array('H')
part_key_shift = 32 - part_exp
part_count = 2 ** part_exp
sha(data).digest())[0] >> self.partition_shift
shuffle(part_list)
Ip = 10.1.0.1
zone = 1
weight = 3.0
class = 1](https://image.slidesharecdn.com/pisa2-140527013013-phpapp01/85/Pisa-20-320.jpg)
![Beolink.org
22
Data Distribution
Proximity base
http://highlyscalable.wordpress.com/2012/09/18/distributed-algorithms-in-nosql-databases/
node_ids = [master_node]
zones = [self.nodes[node_ids[0]]]
for replica in xrange(1, replicas):
while self.part_list[part_id] in node_ids :
part_id += 1
if part_id >= len(self.part_list):
part_id = 0
node_ids.append(self.part_list[part_id])
return [self.nodes[n] for n in node_ids]
Part Serv
1 xxxx
2 yyyyy
3 zzzzz
4
5
…
Partition one will be
also in node 2 and 3 ,
the master node is
always the first](https://image.slidesharecdn.com/pisa2-140527013013-phpapp01/85/Pisa-22-320.jpg)
Pisa
- 1. Beolink.org Pisa Block Distributionand Replication Framework Fabrizio Manfredi Furuholmen Federico Mosca
- 2. Beolink.org Buzzwords 2014 2 Agenda Introduction Overview Problem Common Pattern Implementation Data placement Data Consistency Cluster Coordination Data Transmission
- 3. Beolink.org Block Storage Devices 3 Pisa isa simple block data distribution and Replication Framework on a wide range of node New Node Transfer New Node New Node Node Data Block Key Data [Hash]
- 4.
- 5. Beolink.orgWhat is it? 5 RestFS is High scalable, high available network object storage
- 6. Beolink.orgFive pylons 6 Objects • Separationbtw data and metadata • Each element is marked with a revision • Each element is marked with an hash. Cache • Client side • Callback/Notify • Persistent Transmission • Parallel operation • Http like protocol • Compression • Transfer by difference Distribution • Resource discovery by DNS • Data spread on multi node cluster • Decentralize • Independents cluster • Data Replication Security • Secure connection • Encryption client side, • Extend ACL • Delegation/Federation • Admin Delegation
- 7. Beolink.org 7 RestFS Key Words RestFS Cell collectionof servers Bucket virtual container, hosted by one or more server Object entity (file, dir, …) contained in a Bucket
- 8. Beolink.orgObject 8 Data Metadata Segments Object Attributes set byuser Properties ACL Ext Properties Block 1 Block 2 Block n Block … HashHashHashHash SerialSerialSerialSerialSerial
- 9. Beolink.org 9 Main Goal … Storageas Lego Brick The infrastructure has to be inexpensive with high scalability and reliability
- 10.
- 11.
- 12.
- 13. Beolink.org 13 CAP theorem According toBrewer’s CAP theorem, it is impossible for any distributed computer system to simultaneously provide all three of Consistency, Availability and Partition Tolerance. You can’t have the three at the same time and get an acceptable latency.
- 14. Beolink.org 14 CAP ACID Atomic: Everything ina transaction succeeds or the entire transaction is rolled back. Consistent: A transaction cannot leave the database in an inconsistent state. Isolated: Transactions cannot interfere with each other. Durable: Completed transactions persist, even when servers restart etc. - Strong consistency for transaction highest priority - Pessimistic - Complex mechanisms - Availability and scaling highest priorities - Weak consistency - Optimistic - Best Effort - Simple and FAST Basic Availability Soft-state Eventual consistency BASE RDBMS NoSQL
- 15. Beolink.orgFirst of all… 15 “Think as a child…”
- 16. Beolink.orgSecond … 16 “There isalways a failure waiting around the corner” *Werner Vogel
- 17.
- 18. Beolink.org 18 Data Placement Better Distribution= partitioning Parallel operation = parallel stream/multi core
- 19. Beolink.org 19 Data Distribution: DHT Distributed Hash Table Blocks are distributed in partitions Partition are identify by an hash prefix Partition hosted in servers 19 Part id Node id 1 2 2 … Node id Node 1 obj 2 obj 0000010000 Key (hash) Partition id
- 20. Beolink.orgData Distribution Zero HopHash (Consistent HASH) - Partition location with 0 hops - 1% capacity added and 1% moved Node • Zone • Weight Partition , array list (FIXED) : • Position = kex prefix • Value = node id Shuffle Avoid sequential allocation Part_list = array('H') part_key_shift = 32 - part_exp part_count = 2 ** part_exp sha(data).digest())[0] >> self.partition_shift shuffle(part_list) Ip = 10.1.0.1 zone = 1 weight = 3.0 class = 1
- 21. Beolink.org 21 Data placement Vnode baseClient base Replication
- 22. Beolink.org 22 Data Distribution Proximity base http://highlyscalable.wordpress.com/2012/09/18/distributed-algorithms-in-nosql-databases/ node_ids= [master_node] zones = [self.nodes[node_ids[0]]] for replica in xrange(1, replicas): while self.part_list[part_id] in node_ids : part_id += 1 if part_id >= len(self.part_list): part_id = 0 node_ids.append(self.part_list[part_id]) return [self.nodes[n] for n in node_ids] Part Serv 1 xxxx 2 yyyyy 3 zzzzz 4 5 … Partition one will be also in node 2 and 3 , the master node is always the first
- 23. Beolink.org 23 Data Consistency http://highlyscalable.wordpress.com/2012/09/18/distributed-algorithms-in-nosql-databases/ To avoidACID implementation but to guarantee the consistency some solution leave to the client the ownership of the algorithm.
- 24. Beolink.org 24 Data Consistency http://highlyscalable.wordpress.com/2012/09/18/distributed-algorithms-in-nosql-databases/ Tunable trade-offsfor distribution and replication (N, R, W) The Read operation is implemented with hash check
- 25.
- 26. Beolink.org 26 Cluster Coordination Epidemic (Gossip) epidemic:anybody can infect anyone else with equal probability O(logn) http://www.cis.cornell.edu/IAI/events/Gossip_Tutorial.pdf Periodic anti-entropy exchanges among nodes ensure that they eventually converge, even if updates are lost. Arbitrary pairs of replicas periodically establish contact and resolve all differences between their databases. Hash reduce the volume of data exchanged in the common case.
- 27. Beolink.org 27 Cluster Coordination Table Items(RoutingTable) • Node table list • Partition 2 Node List Bootstrap • DNS name or IP at startup • DNS Lookup (SRV) • Multicast Transfer Type • Complete transfer • Resync by Diff (Merkel Tree) • Notification for a single change • Join Node • Leave Node • Partition owner Part Serv 1 xxxx 2 … 3 4 5 … Node ID Object 1 xxxx 2 … 3 4 5 … Segment hash 1-100 xxxx 101-200 … …
- 28. Beolink.orgCluster Coordination 28 Node XNew Node Z Bootstrap Part Serv X Z .. … Notify of new node Partition claim x Table Change Notification via Gossip Node Y Accept Client Request part x Return New Owner Requestpartx Returndata In case the date is not present in the new node the new node act as a proxy (Lazy trnasfer)
- 29. Beolink.org 29 Transport Protocol ZeroMQ and MessagePack(RPC) Cluster Communications Client Data transfer Partition replication/Relocation
- 30.
- 31. Beolink.org 31 Next http://www.cs.rutgers.edu/~pxk/417/notes/23-lookup.html Chord Space base/multi dimension New data distribution model Chord/Cluster Node Vector clock Rebalance, handover partition (weight change) Locking WAN area network replication (Async) Config Replication (pub/sub, event) Server Priority …
- 32.
Editor's Notes
- #3 The session devided in three main parts, a small introduction on ecosystem, the presentation of the Goals and architecture of Restfs and a small demo of same basic capabilties of the RestFS
- #7 We devide the design in 5 areas
- #8 Now I will describe a little bit more RestFS, functionality and architcture
- #14 The session devided in three main parts, a small introduction on ecosystem, the presentation of the Goals and architecture of Restfs and a small demo of same basic capabilties of the RestFS
- #16 The principle used for all decicion in design and to identify the right or best solution is base on the idea behind hadoop or GFS and it is
- #17 The principle used for all decicion in design and to identify the right or best solution is base on the idea behind hadoop or GFS and it is