Download as PDF, PPTX
More Related Content
Similar to Large Scale Distributed Storage Systems in Volunteer Computing - Slide
Large Scale Distributed Storage Systems in Volunteer Computing - Slide
- 1. Large-Scale Decentralized Storage Systemsused by Volunteer Computing Final Presentation G7: Arinto, Diego, Enkhjin, Iuliia
- 2. Outline ● Problem Definition ●Volunteer Computing (VC) ● Decentralized Storage Systems (DSS) ● Systems Review ● State-of-the-Art DSS used by VC ● Conclusions
- 3. Problem Definition ● DecentralizedStorage Systems ● Volunteer Computing ● DSS + VC = ?
- 4.
- 5.
- 6. Volunteer Computing ● Group of computing resources ● Voluntary share the resource ● Incentive volunteers!! ● Trust between peers ● Example: ○ SETI@home ○ Storage@home ○ Folding@home ● However, data is centrally stored
- 7.
- 8.
- 9. Decentralized Storage Systems Review FocusCriteria: AV = availability SC = scalability ECO = eventual consistency P = performance SE = security
- 10. Decentralized Storage SystemsReview System Description Focus Large scale persistent P2P storage system SE Farsite P2P storage system based on Dhash table ECO, AV Ivy logs P2P DHT-based storage system that used ECO OverNet XOR-based metrics for routing Large scale persistent P2P storage system SC, SE, AV PAST Highly scalable P2P storage system based on SC, ECO Pastis trust Big, distributed, fault tolerant hashtable AV Voldemort Globally persistent DSS for any-file-anywhere SC, SE OceanStore
- 11. Decentralized Storage Systems Review System Description Focus SC, AV, P Glacier Highly durable DSS for archiving, no remote write P2P storage system. Automatically manage its AV, SC Total Recall availability in dynamic changing environment DSS with no point of failure to store structured SC, AV, ECO Cassandra data SC, AV Riak Dynamo-inspired NOSQL storage system. P, AV, ECO Dynamo Large scale DSS developed by Amazon Secure DSS and BitTorrent-like file swarming SE, AV Attic techniques to serve data and manage load. P, ECO Squirrel Decentralized Web Cache
- 12. State-of-The-Art DSS- Characteristics ● Readand Write Access ● Fault Tolerance and Replication Techniques ● Availability and Symmetry ● Incentives ● Video/Images
- 13.
- 14. State-of-The-Art DSS- Suggestions forIncentives Incentives based on amount of storage dedicated, computing resources volunteered: ● Credit based system ● Printable Certificates ● Government tax break, possible in countries with significant amount of nodes
- 15. State-of-The-Art DSS- Challenges ● Providingthe right Incentive ● Security ● Integration into current VC environment
- 16. Conclusions ● VC datastorage/distribution issues. ● Survey on DSS ○ Classification ● Integration VC systems - DSS ○ Identify characteristics ○ Valid and Useful / Define state-art ○ Challenges ● Tradeoffs ○ functionality and complexity
- 17. Work Effort Arinto Murdopo= 100% Diego Montero = 100% Enkhjin Bayarsaikhan = 100% Iuliia Proskurnia = 100%
- 18.
- 19. Decentralized Storage Systems ● Decentralization ● Consistent Hashing ● Active/Passive ● Availability ○ Fault-Tolerance Replication ● Failure-handling ● Consistency ○ Eventual consistency (gossip-based) ● Logging ● Scalability ● Ring locality