2. Who Am I?
Neil Mackenzie
Azure Architect @ Satory Global
Windows Azure MVP
Blog: http://convective.wordpress.com/
Twitter: @mknz
Book:
Microsoft Windows Azure Development Cookbook
3. Content
Windows Azure Platform
Motivation for SQL Database Federations
Overview of SQL Database Federations
Transact-SQL Statements
DMVs
Fan-out queries
4. Windows Azure
Cloud Services:
Platform-as-a-Service
Web roles, worker roles
Infrastructure-as-a-Service
Virtual Machines, SQL Server 2012
Connectivity
Endpoints, Virtual Network, Service Bus
Storage
Windows Azure Storage Service
Windows Azure SQL Database
5. SQL Database
Database-as-a-Service
Multi-tenanted
Priced in GB/month
Database
One primary and two secondary databases
Quorum commit
Maximum size: 150GB
Programmability
TDS protocol
SQL Server authentication
Change connection string and go
6. Scalability
Scale up
Use more powerful hardware
Price/performance problem
Fails at internet scale
Scale out
Use commodity hardware
Cost-effective way to increase performance
Resilient towards failure
7. Scale-Out Data
Use multiple databases (shards)
Sharding provides:
Larger data sizes
Higher performance
Issues:
Data distribution
Connection routing
9. Federation Members
Federation member
Federated data distributed by distribution key range
Error to insert or update distribution key outside range
Member databases can have:
different sizes
different schemas
Distribution key in:
Clustered index
Each unique index
10. Table Types
Federated tables
In federation members
CREATE TABLE ( … )
FEDERATED ON (CustomerId = custId)
Reference tables
In federation members
CREATE TABLE ( … )
Common tables
In root database
11. Federation Operations
Transact SQL support for federations:
CREATE FEDERATION
USE FEDERATION
ALTER FEDERATION
DROP FEDERATION
12. CREATE FEDERATION
CREATE FEDERATION federation_name
(distribution_name <data_type> RANGE)
Data types:
INT
BIGINT
UNIQUEIDENTIFIER
VARBINARY(n) - n<=900
Root database can support several federations
e.g. – customer federation, product federation
13. USE FEDERATION
USE FEDERATION ROOT WITH RESET
Routes connection to root database
USE FEDERATION federation_name
(distribution_name = value)
WITH RESET, FILTERING={ON|OFF}
Routes connection to appropriate federation member
FILTERING= OFF allows connection to be used for any data in the
federation member
FILTERING=ON restricts connection to a specified distribution
key
14. ALTER FEDERATION: SPLIT
Range: 100 400 500
ALTER FEDERATION CustomerFederation
SPLIT AT (CustomerId = 200)
Range: 100 200 400 500
15. ALTER FEDERATION: DROP
100 200 400 500
ALTER FEDERATION CustomerFederation
DROP AT (LOW CustomerId = 200)
100 200 400 500
ALTER FEDERATION CustomerFederation
DROP AT (HIGH CustomerId = 200)
100 200 400 500
16. Dynamic Management Views
Definition
e.g. sys.federation_member_distributions
History
e.g. sys.federation_member_distribution_history
Operations
e.g. sys.dm_federation_operations
Operation errors
e.g. sys.dm_federation_operation_errors
17. sys.federation_member_distributions
Columns
federation_id int
member_id int (database id)
Distribution_name sysname
Range_low sqlvariant
Range_high sqlvariant
Range:
includes Range_low
excludes Range_high
18. Security Principal
Root database
CREATE USER user_name FROM LOGIN login_name
Federation member
CREATE USER user_name
Security is otherwise like SQL Database
19. Fan-Out Queries
Loop over:
sys.federation_member_distributions.Range_low
Perform queries in parallel
Two-step process
Member query
Summary query (if necessary)
20. Summary
SQL Database Federations
Sharding-as-a-Service
Elastic scalability for SQL Database databases
Management support
Developer support
21. More Information
MSDN Wiki
http://bit.ly/A7sUdo
Cihan Biyikoglu blog:
http://blogs.msdn.com/b/cbiyikoglu/
Post based on this presentation:
http://bit.ly/wqD4Xo
Editor's Notes
Performance / database sizeNew pricing model complicates things since price is not linear in DB sizeFlavorous example
Data Distribution:Range partitioningRound robinHash bucketRouting:Database discoveryConnection managementFlavorusTo support the spike in first-day ticket sales, Flavorus used the Windows Azure platform to host 750 web role instances of the Jetstream application and 550 SQL Azure databases. The application wrote chunks of customer data to multiple sharded SQL Azure databases in parallel. Jetstream was live for two days, after which the company returned the event data to its on-premises servers and continued selling tickets.http://www.microsoft.com/casestudies/Case_Study_Detail.aspx?CaseStudyID=4000011072
A range distribution can be affected by:Data skew – data concentrated in one federation memberOperational skew – data operations concentrated in one federation member
FederatedReferenceCommonIndividual members contain distinct schema
DB_NAME() provides the name of the current federation member.FEDERATION_FILTERING_VALUE(DistributionName) provided the current FILTERING=ON valuehttp://msdn.microsoft.com/en-us/library/windowsazure/hh597465.aspxFederation atomic unit is explicitly not a security boundaryError to insert/update data in wrong member
Only valid when connected to root databaseOrdering of uniqueidentity:00000000-0000-0000-0000-080000000000http://sqlblog.com/blogs/alberto_ferrari/archive/2007/08/31/how-are-guids-sorted-by-sql-server.aspx