Megastore is a storage system developed by Google to meet the requirements of large-scale, interactive online services. It blends the scalability of NoSQL with the convenience of SQL by using Bigtable for data storage and a modified Paxos protocol for ACID transactions. Megastore provides fully serializable transactions within partitions of data and consistency guarantees while maintaining high availability.

1. Megastore
Rob Keisler
CSCI 638 -- Summer 2011

2. What is Megastore?
● A storage system developed by Google to meet the
requirements of today's interactive online services
● Blends the scalability of a NoSQL datastore with
the convenience of a traditional RDBMS
○ Bigtable for logs and data
○ Modified Paxos Logic for ACID
● Provides both consistency guarantees and high availability
● Provides fully serializable ACID semantics within fine-
grained partitions of data

3. Architecture

4. Entity Groups
Example Cross Entity Group
Application Entity Groups
Operations
Email Account None (external to the system)
Users, Blogs,
Blogs Access Control, Notifications
Posts
Globe Divisions Operations that span multiple
Maps
(Patches) patches

5. Entity Groups

6. Data Model
CREATE SCHEMA PhotoApp;
CREATE TABLE User {
required int64 user_id;
required string name;
} PRIMARY KEY(user_id), ENTITY GROUP ROOT;
CREATE TABLE Photo {
required int64 user_id;
required int32 photo_id;
required int64 time;
required string full_url;
optional string thumbnail_url;
repeated string tag;
} PRIMARY KEY(user_id, photo_id),
IN TABLE User,
ENTITY GROUP KEY(user_id) REFERENCES User;
CREATE LOCAL INDEX PhotosByTime
ON Photo(user_id, time);
CREATE GLOBAL INDEX PhotosByTag
ON Photo(tag) STORING (thumbnail_url);

7. Transactions and Concurrency Control

8. Reads

9. Writes

10. Questions?
http://research.google.com/pubs/pub36971.html