Megastore

Megastore Rob KeislerCSCI 638 -- Summer 2011

What is Megastore?● A storage system developed by Google to meet the requirements of todays interactive online services● Blends the scalability of a NoSQL datastore with the convenience of a traditional RDBMS ○ Bigtable for logs and data ○ Modified Paxos Logic for ACID● Provides both consistency guarantees and high availability● Provides fully serializable ACID semantics within fine- grained partitions of data

Architecture

Entity Groups Example Cross Entity Group Application Entity Groups Operations Email Account None (external to the system) Users, Blogs, Blogs Access Control, Notifications Posts Globe Divisions Operations that span multiple Maps (Patches) patches

Entity Groups

Data ModelCREATE SCHEMA PhotoApp;CREATE TABLE User { required int64 user_id; required string name;} PRIMARY KEY(user_id), ENTITY GROUP ROOT;CREATE TABLE Photo { required int64 user_id; required int32 photo_id; required int64 time; required string full_url; optional string thumbnail_url; repeated string tag;} PRIMARY KEY(user_id, photo_id), IN TABLE User, ENTITY GROUP KEY(user_id) REFERENCES User;CREATE LOCAL INDEX PhotosByTime ON Photo(user_id, time);CREATE GLOBAL INDEX PhotosByTag ON Photo(tag) STORING (thumbnail_url);

Transactions and Concurrency Control

Writes

Questions?http://research.google.com/pubs/pub36971.html

Megastore

by robjk

Accessibility

Categories

Upload Details

Usage Rights

Statistics

Megastore Webinar Transcript