Megastore - ID2220 Presentation

1,065 views

Published on

Paper presentation for Advanced Topics in Distributed System. Original paper can be found here -> http://research.google.com/pubs/pub36971.html

Published in: Education
0 Comments
1 Like
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total views
1,065
On SlideShare
0
From Embeds
0
Number of Embeds
1
Actions
Shares
0
Downloads
38
Comments
0
Likes
1
Embeds 0
No embeds

No notes for slide

Megastore - ID2220 Presentation

  1. 1. Megastore Providing Scalable, Highly Available Storage for Interactive Services Paper by Jason Barker et al. Presented by Arinto Murdopo arinto@kth.se7/11/2012 1
  2. 2. OutlineMotivationMegastore:  Features  Scalability  Availabilty  Putting them all togetherObservationConclusions7/11/2012 2
  3. 3. Motivation  Conflicting requirements • RDBMS – easy to use, but not scale • NoSQL – scale, but not easy to use  Interactive online services • Highly available and fast response time7/11/2012 3
  4. 4. Here comes Megastore easy to use • ACID semantics scalable • data partitioning highly available • synchronous replication through modified Paxos7/11/2012 4
  5. 5. Easy to use - Featurescost-transparent APIs • No API for joins • Joins are implemented in application codedata model • schema, table (entity), property • entity clustering • indexes: local, global • Bigtable column name == Megastore table name and property name, i.e User.name7/11/2012 5
  6. 6. Easy to use - Featurestransactions and concurrency control • Bigtable for concurrency control • transaction lifecycle: read, application logic, commit, apply, clean upothers • backup system of transaction logs • encryption7/11/2012 6
  7. 7. ScalableScale the replication scheme Data partitioning • Entity group concept Data locality • Entity group locality • Bigtable instances locality7/11/2012 7
  8. 8. Entity GroupsEntity is like instance of table.Entity group is group of entities. i.e Email Application • Email account Blog Application • User Profile • Blog post + metadata • Blog unique name7/11/2012 8
  9. 9. Entity Groups7/11/2012 9
  10. 10. Highly AvailableReplicate mutations of write-ahead log inside entity groupsusing modified Paxos, but let’s revisit original Paxos…7/11/2012 10
  11. 11. Modified Paxos – Fast ReadsRead in original Paxos7/11/2012 11
  12. 12. Modified Paxos – Fast ReadsContact Coordinator and read locally if possible7/11/2012 12
  13. 13. Modified Paxos – Fast WritesSkip “prepare” stage in subsequent write of sameleader, provided no write from other writers7/11/2012 13
  14. 14. Modified Paxos – New ReplicaTypes  Full Replicas  all replicas that we have seen until now  Witness Replicas  are able to vote  store but do not apply write-ahead logs  do not store entity data  Read-only Replicas  are not able to vote  snapshots of entity data7/11/2012 14
  15. 15. Putting them all togetherMegastore Architecture7/11/2012 15
  16. 16. Reads Query Local Find Position Catchup Validate Query Data7/11/2012 16
  17. 17. Writes Accept Leader Prepare Accept Invalidate Apply7/11/2012 17
  18. 18. Observation - Availability7/11/2012 18
  19. 19. Observation – Latency7/11/2012 19
  20. 20. Conclusion  Megastore and its motivation  Features of megastore • It has ACID semantics • But need to define entity groups • Need to handle inter-group updates  Scalability and Availability  More experiments are needed7/11/2012 20

×