0
mrys@microsoft.com

MySpace: the Business Problem 223M users 900 Terabytes of data 450 SQL Servers Required (eventual) data  consistency a...
MySpace’s Data Consistency Problem    1-1000     1001-2000   2001-3000   3001-4000   4001-5000   5001-6000
MySpace’s Solution Propagate data changes from one DB to  other DBs using reliable, async Message  Service (Service Broke...
MySpace’s Service Dispatcher Coordination point between all SQL Servers Load-balanced across 30 SQL Servers Enables mul...
MySpace Architecture1-1000          3001-4000                                               My DB                I change ...
Many other customers using similarpatterns Online electronic stores (cannot give names  ) Travel reservation systems (e...

Social                                                          Social                                                    ...

•••
Partitioned over 100 SQL Azure DBs                                                                                  Find F...
• Fanout: Parallel calls to   multiple database   partitions• Quorum: Able to   tolerate a percentage of   request failure...
Statistics per game





Readable                                         Replica                        Primary Shard                             ...

   Federation                                                              Azure DB with Federation Root        Represent...
•••

    http://www.microsoft.com/casestudies/Case_Study_Detail.aspx?CaseSt    udyID=4000008310                           htt...
Connection:     Server=az1cl321.db.windows.net;     Database=MyDB;     User=AppUser;     Passwd=****;                     ...
Federation with a Single Shard                                                    Existing Database                       ...
Introducing Two Connection Modes • Filtered Connection    – Guarantees that any queries or DML will produce the      same ...
Create Schema on Member: Management Connection                                                                            ...
DDL CREATE TABLE customer (   c_id bigint PRIMARY KEY,   …   )   FEDERATED ON (customer_id=c_id); CREATE TABLE order (   i...
More Detail • Supported data types for federation key : bigint, int, GUID, and varbinary   (900)     – Only range partitio...
Connect to Atomic Unit: Filtered                                                                       Existing Database  ...
More on Connection Filtering • Most operations behave differently in filtered vs   unfiltered connections • Connection fil...
Support Matrix                    Connection Type Filtered Unfiltered Named              Operation                        ...
Splitting a Member                                                                          Existing Database             ...
Two New Members                                                                              Existing Database            ...
Two New Members                                                                                Existing Database          ...
Scaling with SQL Server and SQL Azure Federations
Scaling with SQL Server and SQL Azure Federations
Scaling with SQL Server and SQL Azure Federations
Upcoming SlideShare
Loading in...5
×

Scaling with SQL Server and SQL Azure Federations

2,738

Published on

Slides for my presentation at the Seattle Hadoop/NoSQL Meetup (http://www.meetup.com/Seattle-Hadoop-HBase-NoSQL-Meetup/events/40509972/).
These slides are based on this earlier presentation: http://www.slideshare.net/MichaelRys/scaling-with-sql-server-and-sql-azure-federations.

Published in: Education, Technology
0 Comments
1 Like
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total Views
2,738
On Slideshare
0
From Embeds
0
Number of Embeds
1
Actions
Shares
0
Downloads
47
Comments
0
Likes
1
Embeds 0
No embeds

No notes for slide
  • 3000 web servers WS2003 IISThere is an app tier between the data and the web tier (not shown to focus on the partitioning aspect).A component in the app tier uses the partitioning function to connect to the relevant database in the data tier
  • For multicast/broadcast and feedback functionality: messages include a header (as part of the payload) that contains the target list and the initiator database
  • MyDB sends a message with my status change and a target list specifying the DBs that store my friends data.The Service Dispatcher forwards the message these DBs.Each DB processes the message updating my status in a partitioned tableTalk about other MySpace scenarios: Clean state (e.g. on account close) Deploy business logic (stored procedures)
  • 250 Router Services
  • Note: Big-sized companies invest resources in building these platforms instead of using existing relational platforms!
  • Note: Big-sized companies invest resources in building these platforms instead of using existing relational platforms!
  • Performance and Scale:Map/Reduce PatternsEventual consistency (trade-off due to CAP)ShardingCachingAutomate management Lifecycle:Elastic Scale on demand (no need to pay for resources until needed)Automatic Fail-overScalable Schema version rolloutPerf troubleshootingAuto alertingAuto loadbalancingAuto resourcing (e.g., auto splits based on policies)Declarative policy-based management
  • Code First and revise quicklyWorking software over comprehensive documentationResponding to change over following a planApplication-model first (before database) Dictates the data model and queriesFlexible data modelsNo a priori modeling: Data first, schema later/Open SchemaKey/Value storesReduced impedance mismatch: JSON, XML, YAMLYou don’t know exactly what you are looking forMap/Reduce for adhoc analysisProvide Search across all your data instead of just queryLower Pain of adoption and maintenance From code to deployment & “monetization” of data, services, apps and tenantsRich Services out of the BoxData and services mashupEasy troubleshooting of deployed appsNo DB or OS Admin telling me what to do
  • Update syntax
  • Transcript of "Scaling with SQL Server and SQL Azure Federations"

    1. 1. mrys@microsoft.com
    2. 2. 
    3. 3. MySpace: the Business Problem 223M users 900 Terabytes of data 450 SQL Servers Required (eventual) data consistency across databases
    4. 4. MySpace’s Data Consistency Problem 1-1000 1001-2000 2001-3000 3001-4000 4001-5000 5001-6000
    5. 5. MySpace’s Solution Propagate data changes from one DB to other DBs using reliable, async Message Service (Service Broker) And also used for
    6. 6. MySpace’s Service Dispatcher Coordination point between all SQL Servers Load-balanced across 30 SQL Servers Enables multicast/broadcast functionality 18,000 ~2k msgs/sec per dispatcher SQL Server
    7. 7. MySpace Architecture1-1000 3001-4000 My DB I change gets updated my status Service Service TX3 Broker TX2 TX1 Dispatcher Service userId=1024 Broker2001-3000 Service 1001-2000 Broker TX4 TX54001-5000 5001-6000 Web Tier Data Tier
    8. 8. Many other customers using similarpatterns Online electronic stores (cannot give names ) Travel reservation systems (e.g. Choice International)
    9. 9. 
    10. 10. Social Social Networks Network Management s Ops Services Data Backend Data Backend Services Data Backend Services Services MSN MSN GamesGames l Web Portal Front Door GameBar Router WLM Services WLM Games HostGames Web Portal Bing Bing Auth Games STS STSGames l Web Portal Azure Data Centers
    11. 11. 
    12. 12. •••
    13. 13. Partitioned over 100 SQL Azure DBs Find Friends’ Profiles Social Get my Profile Social Services User … DB Service Publish feed, read feed Find Friends’ Profiles Get Friends highscores Gamer Last Played Gamer Services Favorites STS Services Game Preferences STS Leaderb … Social Leaderboards DB oard Partitioned over 298 SQL Azure DBs Game Game Disable/Enable Ingestion Games from Front Door Write user specific game infos Ingestion accessing services Router Services Game Game binaries User … DB Catalog Game metadata 250 instances Partitioned over 100 SQL Azure DBs 250 instances
    14. 14. • Fanout: Parallel calls to multiple database partitions• Quorum: Able to tolerate a percentage of request failures during Fanout• Retry: Retry on database requests error
    15. 15. Statistics per game
    16. 16. 
    17. 17. 
    18. 18. 
    19. 19. 
    20. 20. 
    21. 21. Readable Replica Primary Shard Readable Replica Dynamic OLAP Workloads (Scale-out queries,OLTP Workloads often using(mostly touching 1 to eventual consistent Readable scale-outlow number of shards) Replica frameworks like Hadoop) Primary Shard Readable Replica Readable Replica Primary Shard Readable Replica
    22. 22. 
    23. 23.  Federation Azure DB with Federation Root Represents the data being sharded Federation Root Federation Directories, Federation Users, Database that logically houses federations, contains federation meta data Federation Distributions, … Federation Key Value that determines the routing of a piece of data (defines a Federation Distribution) Federation Member (aka Shard) Federation “Orders_Fed” A physical container for a set of federated (Federation Key: CustomerID) tables for a specific key range and reference tables Member: PK [min, 100) Federated Table Table that contains only atomic units for the AU AU AU member’s key range PK=5 PK=25 PK=35 Reference Table Non-sharded table Atomic Unit Member: PK [100, 488) All rows with the same federation key value: always together! AU AU AU Connection PK=105 PK=235 PK=365 Gateway Member: PK [488, max) AU AU AU Sharded PK=555 PK=2545 PK=3565 Application 25
    24. 24. •••
    25. 25. 
    26. 26.  http://www.microsoft.com/casestudies/Case_Study_Detail.aspx?CaseSt udyID=4000008310 http://cacm.acm.org/magazines/2011/6/108663- scalable-sql http://download.microsoft.com/download/9/E/9/9E9F240D-0EB6-472E- B4DE- 6D9FCBB505DD/Windows%20Azure%20No%20SQL%20White%20Paper.p df http://blogs.msdn.com/b/cbiyikoglu/archive/2011/03/03/nosql-genes-in- sql-azure-federations.aspx mrys@microsoft.com http://sqlblog.com/blogs/michael_rys/default.aspx
    27. 27. Connection: Server=az1cl321.db.windows.net; Database=MyDB; User=AppUser; Passwd=****; Gateway CREATE FEDERATION Existing Database CREATE FEDERATION sales (customer_id bigint RANGE)30
    28. 28. Federation with a Single Shard Existing Database CREATE FEDERATION sales (customer_id bigint RANGE) Database root contains: sales • Federation root = DB level object containing federation scheme Gateway • Federation users • Federation metadata incl. federation Connection: mapServer=az1cl321.db.windows.net; Range: Min...MaxDatabase=MyDB;User=AppUser;Passwd=****; Federation Member 31
    29. 29. Introducing Two Connection Modes • Filtered Connection – Guarantees that any queries or DML will produce the same results independent of changes to the physical layout of the federation members – Scoped to an “Atomic Unit” • Unfiltered Connection – Scoped to a Federation Member – Management Connection 32
    30. 30. Create Schema on Member: Management Connection Existing Database sales Gateway USE FEDEDERATION sales (customer_id=0) Connection: WITH FILTERING=OFF, RESET;Server=az1cl321.db.windows.net; CREATE TABLE … Range: Min...Max Customer Order ProductDatabase=MyDB;User=AppUser;Passwd=****; Federation Member © 2011 Microsoft Corporation. Microsoft Materials - Confidential. All rights reserved. CITA # 33 MSFT101120_A
    31. 31. DDL CREATE TABLE customer ( c_id bigint PRIMARY KEY, … ) FEDERATED ON (customer_id=c_id); CREATE TABLE order ( item_num int, customer_id bigint, date_sold datetime2, …, CONSTRAINT PK_Order PRIMARY KEY (item_num, customer_id, date_sold), CONSTRAINT FK_Cust FOREIGN KEY customer_id REFERENCES customer (customer_id) ) FEDERATED ON (customer_id=customer_id); CREATE TABLE product ( product_name varchar(100) NOT NULL, unit_price money, item_num int PRIMARY KEY, … ); 34
    32. 32. More Detail • Supported data types for federation key : bigint, int, GUID, and varbinary (900) – Only range partitioning • Federation key must be part of unique index • Foreign key constraints only allowed between federated tables and from federated table to reference table • Not all Azure programmability features supported – Sequence, timestamp • Additional surface area restrictions – Indexed views, drop database (members) • Schemas are allowed to diverge over time – Furthermore, in v1, schema updates to existing members must be done in each member (where the change is needed) • USE FEDERATION “rewires a connection” – Connection is reestablished – All existing settings and context of the connection is lost (sp_reset_connection) – Must be in a batch by itself 35
    33. 33. Connect to Atomic Unit: Filtered Existing Database When using into a specific key value, SELECT will only return sales records from federated tables that match that value. It will still return Gateway all records from non-federated tables. USE FEDEDERATION sales (customer_id=3) Connection: Inserts and UPDATES operating WITH FILTERING=ON, RESET; outside of the value will fail.Server=az1cl321.db.windows.net; Range: Min...Max customer order product 3Database=MyDB; SELECT * from customer 3User=AppUser;Passwd=****; SELECT * from product SELECT * from order Federation Member 36
    34. 34. More on Connection Filtering • Most operations behave differently in filtered vs unfiltered connections • Connection filtering is a property of the session – Filter injected dynamically at runtime – Cannot inspect source code to determine how it behaves • E.g., running stored proc written for filtered mode on unfiltered connection could lead to unintended results • There are several operations that will not work in filtered connection in v1 – DDL, DML on reference tables, … • Fan-out, bulk operations not efficient in filtered mode – For now, filter=off is our best offer 37
    35. 35. Support Matrix Connection Type Filtered Unfiltered Named Operation (unfiltered) Dynamic SELECT P P P DML* (federated tables) P P P DML* (reference tables) X P P DDL X P P Views (not indexed) P P P UDF - activate P P P Stored Proc - activate P P P Trigger (all modes) - activate P P P CREATE/UPDATE Stats X P P Bulk Ops openrowset bulk, bcp, bulk insert X P P * not including SELECT & modules ^ autostats will work on all connections System stored procs, intrinsics will be unaffected (run unfiltered) 38
    36. 36. Splitting a Member Existing Database ALTER FEDERATION sales sales SPLIT AT (customer_id=50) Gateway USE FEDERATION ROOT Connection: WITH RESET Using to theServer=az1cl321.db.windows.net; federation ROOT will pop you out of a member back Range: Min...Max into the database that hosts the customer order product federation 3 3Database=MyDB;User=AppUser;Passwd=****; 40 58 58 58 Federation Member 39
    37. 37. Two New Members Existing Database ALTER FEDERATION sales sales SPLIT AT (customer_id=50) Gateway USE FEDEDERATION ROOT WITH RESET Connection:Server=az1cl321.db.windows.net; Range: Min...50 Range: 51...Max customer order product customer order product 3Database=MyDB; 3User=AppUser; 58Passwd=****; 40 58 58 Federation Member Federation Member 40
    38. 38. Two New Members Existing Database sales Gateway USE FEDEDERATION sales (customer_id=40) Connection: WITH FILTERING=ON, RESET;Server=az1cl321.db.windows.net; Range: Min...50 Range: 51...Max customer order product customer order productDatabase=MyDB; SELECT *User=AppUser; 58Passwd=****; from customer 40 40 58 58 SELECT * from order 41
    1. A particular slide catching your eye?

      Clipping is a handy way to collect important slides you want to go back to later.

    ×