Your SlideShare is downloading. ×
0
Megastore            Providing Scalable, Highly Available              Storage for Interactive Services                   ...
OutlineMotivationMegastore:   Features   Scalability   Availabilty   Putting them all togetherObservationConclusio...
Motivation   Conflicting requirements       •    RDBMS – easy to use, but not scale       •    NoSQL – scale, but not eas...
Here comes Megastore easy to use     • ACID semantics scalable     • data partitioning highly available     • synchrono...
Easy to use - Featurescost-transparent APIs     • No API for joins     • Joins are implemented in application codedata m...
Easy to use - Featurestransactions and concurrency control     • Bigtable for concurrency control     • transaction lifec...
ScalableScale the replication scheme Data partitioning   • Entity group concept Data locality   • Entity group locality ...
Entity GroupsEntity is like instance of table.Entity group is group of entities. i.e  Email Application            • Emai...
Entity Groups7/11/2012       9
Highly AvailableReplicate mutations of write-ahead log inside entity groupsusing modified Paxos, but let’s revisit origina...
Modified Paxos – Fast ReadsRead in original Paxos7/11/2012                 11
Modified Paxos – Fast ReadsContact Coordinator and read locally if possible7/11/2012                                      ...
Modified Paxos – Fast WritesSkip “prepare” stage in subsequent write of sameleader, provided no write from other writers7/...
Modified Paxos – New ReplicaTypes   Full Replicas        all replicas that we have seen until now   Witness Replicas   ...
Putting them all togetherMegastore Architecture7/11/2012                   15
Reads  Query Local  Find Position    Catchup     Validate   Query Data7/11/2012         16
Writes   Accept Leader       Prepare            Accept      Invalidate            Apply7/11/2012            17
Observation - Availability7/11/2012                    18
Observation – Latency7/11/2012               19
Conclusion   Megastore and its motivation   Features of megastore      •     It has ACID semantics      •     But need t...
Upcoming SlideShare
Loading in...5
×

Megastore - ID2220 Presentation

533

Published on

Paper presentation for Advanced Topics in Distributed System. Original paper can be found here -> http://research.google.com/pubs/pub36971.html

Published in: Education
0 Comments
1 Like
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total Views
533
On Slideshare
0
From Embeds
0
Number of Embeds
0
Actions
Shares
0
Downloads
30
Comments
0
Likes
1
Embeds 0
No embeds

No notes for slide

Transcript of "Megastore - ID2220 Presentation"

  1. 1. Megastore Providing Scalable, Highly Available Storage for Interactive Services Paper by Jason Barker et al. Presented by Arinto Murdopo arinto@kth.se7/11/2012 1
  2. 2. OutlineMotivationMegastore:  Features  Scalability  Availabilty  Putting them all togetherObservationConclusions7/11/2012 2
  3. 3. Motivation  Conflicting requirements • RDBMS – easy to use, but not scale • NoSQL – scale, but not easy to use  Interactive online services • Highly available and fast response time7/11/2012 3
  4. 4. Here comes Megastore easy to use • ACID semantics scalable • data partitioning highly available • synchronous replication through modified Paxos7/11/2012 4
  5. 5. Easy to use - Featurescost-transparent APIs • No API for joins • Joins are implemented in application codedata model • schema, table (entity), property • entity clustering • indexes: local, global • Bigtable column name == Megastore table name and property name, i.e User.name7/11/2012 5
  6. 6. Easy to use - Featurestransactions and concurrency control • Bigtable for concurrency control • transaction lifecycle: read, application logic, commit, apply, clean upothers • backup system of transaction logs • encryption7/11/2012 6
  7. 7. ScalableScale the replication scheme Data partitioning • Entity group concept Data locality • Entity group locality • Bigtable instances locality7/11/2012 7
  8. 8. Entity GroupsEntity is like instance of table.Entity group is group of entities. i.e Email Application • Email account Blog Application • User Profile • Blog post + metadata • Blog unique name7/11/2012 8
  9. 9. Entity Groups7/11/2012 9
  10. 10. Highly AvailableReplicate mutations of write-ahead log inside entity groupsusing modified Paxos, but let’s revisit original Paxos…7/11/2012 10
  11. 11. Modified Paxos – Fast ReadsRead in original Paxos7/11/2012 11
  12. 12. Modified Paxos – Fast ReadsContact Coordinator and read locally if possible7/11/2012 12
  13. 13. Modified Paxos – Fast WritesSkip “prepare” stage in subsequent write of sameleader, provided no write from other writers7/11/2012 13
  14. 14. Modified Paxos – New ReplicaTypes  Full Replicas  all replicas that we have seen until now  Witness Replicas  are able to vote  store but do not apply write-ahead logs  do not store entity data  Read-only Replicas  are not able to vote  snapshots of entity data7/11/2012 14
  15. 15. Putting them all togetherMegastore Architecture7/11/2012 15
  16. 16. Reads Query Local Find Position Catchup Validate Query Data7/11/2012 16
  17. 17. Writes Accept Leader Prepare Accept Invalidate Apply7/11/2012 17
  18. 18. Observation - Availability7/11/2012 18
  19. 19. Observation – Latency7/11/2012 19
  20. 20. Conclusion  Megastore and its motivation  Features of megastore • It has ACID semantics • But need to define entity groups • Need to handle inter-group updates  Scalability and Availability  More experiments are needed7/11/2012 20
  1. A particular slide catching your eye?

    Clipping is a handy way to collect important slides you want to go back to later.

×