Introduction to SQL Server Cloud Storage Azure

Dr. Eduardo Castro MartínezMicrosoft MVPecastro@mswindowscr.orghttp://comunidadwindows.orghttp://ecastrom.blogspot.com

CloudApplication Cloud Platform Users Developers

SQL Azure CloudApplications Windows AzureWindows Azure AppFabric On-Premises Applications Windows Others

Application Compute Storage Config Fabric … SQL AzureApplications Windows Azure AppFabricWindows Azure Applications Windows Others

VMs VMsHTTP/HTTPS Web Worker Load Role Role Balancer IIS Instance Instance Application Compute Storage Fabric …

Storage Web Role Worker Role Instance Instance Fabric Fabric Agent Agent FabricController

HTTP/ HTTPS Blobs Tables QueuesApplication Compute Storage Fabric …

Tables Provide structured storage. A Table isa set of entities, which contain a set ofpropertiesQueuesBlobsDrives (new) 11

Table Table Table ... Entity Entity Entity ... Storage Property Property PropertyAccounts Name Type Value

1) Receive Web Role Worker Role work main() 4) Do ASP.NET, work WCF, etc. { … } 2) Put 3) Get message in message queue from queue 5) Delete message from queue Queue

Accounts Tables Entities Email =… Name = … Email =… Name = … Genre =… Title = … Genre =… Title = … 21

PartitionKey RowKey Timestamp ReleaseDatePartitionKey RowKey Timestamp ReleaseDate(Category) (Title)(Category) (Title)Action Fast & Furious Fast & Furious … … 2009 2009Action The Bourne Ultimatum … 2007…… … … … … … …Animation Open Season 2 … 2009Animation Open Season 2 … 2009Animation The Ant Bully … 2006Animation The Ant Bully … 2006…PartitionKey … RowKey …Timestamp … ReleaseDate(Category) (Title)ComedyComedy Office Space Office Space … … 1999 1999…… … … …… … …SciFi X-Men Origins: …… 2009 Wolverine… … …… …WarWar Defiance Defiance …… 2008 2008 25

[DataServiceKey("PartitionKey", "RowKey")]public class Movie{ /// <summary> /// Category is the partition key /// </summary> public string PartitionKey { get; set; } /// <summary> /// Title is the row key /// </summary> public string RowKey { get; set; } public DateTime Timestamp { get; set; } public int ReleaseYear { get; set; } public string Language { get; set; } public string Cast { get; set; }} 27

StorageCredentialsAccountAndKey credentials = new StorageCredentialsAccountAndKey( “myaccount", “myKey");string baseUri = "http://myaccount.table.core.windows.net";CloudTableClient tableClient = new CloudTableClient(baseUri, credentials);tableClient.CreateTable(“Movies");TableServiceContext context = tableClient.GetDataServiceContext();CloudTableQuery<Movie> q = (from movie in context.CreateQuery<Movie>(“Movies") where movie.PartitionKey == “Action" && movie.RowKey == "The Bourne Ultimatum" select movie).AsTableServiceQuery<Movie>();Movie movieToUpdate = q.FirstOrDefault();// Update moviecontext.UpdateObject(movieToUpdate);context.SaveChangesWithRetries();// Add moviecontext.AddObject(new Movie(“Action" , movieToAdd));context.SaveChangesWithRetries(); 28

> > > >> > >> (new) > > 29

> > 30

> > > > PartitionKey RowKey Category … (Account name) (Title)> moviesonline 12 Rounds Action … moviesonline A Bug’s Life Animation … 100,000,000 more rows … … … moviesonline Office Space Comedy … moviesonline Platoon War … 50,000,000 more rows … … … moviesonline WALL-E Animation … 31

>>> PartitionKey RowKey Category … (Account name) (Title) Client Client moviesonline 12 Rounds Action … moviesonline A Bug’s Life Animation … 100,000,000 more rows … … … moviesonline Office Space Comedy … moviesonline Platoon War … 50,000,000 more rows … … … moviesonline WALL-E Animation … 32

>>> PartitionKey RowKey (Title) (Category) (Title)> Action Fast & Furious … 10000 more Action movies 10000 more Action movies Action The Bourne Ultimatum … 100000 more Action & & Animation 100000 more Action Animation movies movies Animation Open Season 2 Animation … Open Season 2 100000 more Animation movies Client Client … Animation 100000 more Animation movies The Ant Bully Animation The Ant Bully Comedy Office Space … 1000000more Comedy & & SciFi movies 1000000 more Comedy SciFi movies SciFi Star Trek … 100000 more SciFi & War movies … 100000 more War movies War War Defiance Defiance 33

> > > 34

> > > > PartitionKey Properties (Timestamp) > 2009-11-15 02:00:01 … > 2009-11-15 02:00:11 … > 100000 more rows … 2009-11-17 05:40:01 … 2009-11-17 05:40:01 … 80000 more rows … Applications 2009-11-17 12:30:00 … Client 2009-11-17 12:30:01 … 2009-11-17 12:30:01 … 2009-11-17 12:30:02 … 2009-11-17 12:30:03 … 35

> > PartitionKey Properties > (ID_Timestamp)> 01_2009-10-12 05:10:00 … … 01_2009-11-17 12:30:01 …> 100000 more rows …> 09_2009-11-15 12:31:00 … > … 09_2009-11-17 12:30:22 … 20000000 more rows … 10_2009-10-05 05:10:10 …Applications Client 5000000 more rows … … 15_2009-11-17 12:30:01 … … 900000 more rows … 19_2009-11-17 12:20:02 … 19_2009-11-17 12:30:10 … 36

>>> 37

>>> PartitionKey (Category) RowKey (Title) Action Fast & Furious … 10000 more Action movies Action The Bourne Ultimatum … 100000 more Action & Animation movies Animation Open Season 2 … 100000 more Animation movies Client Animation The Ant Bully Comedy Office Space … 1000000 more Comedy & SciFi movies SciFi Star Trek … 100000 more SciFi & War movies … 100000 more War movies War Defiance 38

>> PartitionKey (Category) RowKey (Title) Rating > Action Fast & Furious 5 > … Returns 1000 rated > 4 999 more movies movies …> … Action and Anim. movies here with rating < 4 … Animation A Bug’s life 2 Partition range boundary hit … 100 more movies < 4 here … Animation The Ant Bully 3 Comedy Are we there yet? 2 … Return continuation More movies here … Client Comedy Office Space 5 … 800000 more movies here … Drama A Beautiful Mind 5 … 1200000 more movies here … War Defiance 4 39

> > > > PartitionKey RowKey Rating> (Category) (Title)> Action Fast & Furious 5 … More movies here … Comedy Office Space 5 … More movies here … Documentary Planet Earth 4 … More movies here Drama Seven Pounds 4 Horror Saw 5 3 Client … More movies here … Music 8 Mile 2 … More movies here … SciFi Star Trek 5 … More movies here … 40

1. Fast > 2. Medium > > 3. Slow > > >> 1 41

Faster> > > Create and maintain own views that help queries> “Or” Predicates > Execute individual query in parallel instead of using “OR”> User Interactive > Cache the result to reduce scan frequency 42

>>> 43

(new)>>> > 44

> > > > >> > > > > 45

> > > > > > > >PartitionKey RowKey Kind TotalRentals Name Address CheckOutOn Title DueOn(AccountID) (Kind_*)… … … … … … … … …Sally A_Sally Account 8 Sally Field Ann Arbor, MISally R_Jaws Rental 2009/11/16 Jaws 2009/11/20Sally R_Taxi Rental 2009/11/16 Taxi 2009/11/20… … … … … … … … … 46

> > > >>>> > Future Feature - Secondary Index>> 47

Accounts Queues Messages 49

Message now has dequeue count 50

CloudQueueClient queueClient = new CloudQueueClient(baseUri, credentials);CloudQueue queue = queueClient.GetQueueReference("test1");queue.CreateIfNotExist();//MessageCount is populated via FetchAttributesqueue.FetchAttributes();CloudQueueMessage message = new CloudQueueMessage("Some content");queue.AddMessage(message);message = queue.GetMessage(TimeSpan.FromMinutes(10) /*visibility timeout*/);//Process the message here …queue.DeleteMessage(message); 52

Producers Consumers P2 C1 1. GetMessage(Q, 30 s)  msg 1 4 3 2 1 30 1 1 0 0 0 C2 P1 2. GetMessage(Q, 30 s)  msg 2 53

Producers Consumers 1 C1 1. GetMessage(Q, 30 s)  msg 1 P2 1 5. C1 crashed 4 3 2 1 3 6. msg1 visible 30 s after Dequeue 0 0 1 2 1 2 1 C2 P1 2. GetMessage(Q, 30 s)  msg 2 3. C2 consumed msg 2 4. DeleteMessage(Q, msg 2) 7. GetMessage(Q, 30 s)  msg 1 54

Producers Consumers 1. Dequeue(Q, 30 sec)  msg 1 P2 C1 5. C1 crashed 10. C1 restarted 11. Dequeue(Q, 30 sec)  msg 1 4 33 1 12. DequeueCount > 2 0 0 3 2 13. Delete (Q, msg1) 1 C2 P1 2 6. msg1 visible 30s after Dequeue2. Dequeue(Q, 30 sec)  msg 2 9. msg1 visible 30s after Dequeue3. C2 consumed msg 24. Delete(Q, msg 2)7. Dequeue(Q, 30 sec)  msg 18. C2 crashed 55

> >> >> >>> > > > 56

> > > > (new)> > > (new)>> (new) 57

A relational DB in the cloud SQL Azure Database Data Hub Others (Future) .NET Services Applications SQL Services Relational database as a service Highly available, automatically maintained Live Services Windows Azure Extension of the SQL Server Data Platform Applications Windows Windows Windows Others Server Vista/XP Mobile

Single Logical Multiple Physical Database Replicas Replic a1 Replic D a2 B Replic a3

Reference Data Sync Data Symmetric Programming Model Data Hub Aggregation• Initial services – core RDBMS capabilities with SQL Azure Database, Data Sync• Future Offerings • Additional data platform capabilities: Reporting, BI • New services: Reference Data

Clear Feedback: “I want a database in the Cloud”Focus on combining the best features of SQL Server running at scale with low friction

Application Application Application Browser Browser ODBC, OLEDB, Application Application REST Client REST Client SQL Client* ADO.Net PHP, Ruby, … Cloud Cloud Evolves HTTP+REST HTTP+REST HTTP HTTP TDS Windows Azure Windows Azure Web App REST (Astoria) Web AppData Center Data Center REST Client ADO.Net + EF SQL Client* REST/SOAP + ACE Model TDS + TSQL Model SDS Current SDS Next * Client access enabled using TDS for ODBC, ADO.Net, OLEDB, PHP-SQL, Ruby, …

SQL Azure Database Others (Future) SQL AzureApplications Windows Azure AppFabricWindows Azure Applications Windows Others

SQL Azure Database TDS DatabaseApplication Database DatabaseSQL Azure Database Others (Future)

SQL Azure Database DatabaseApplication Max database Database size is 50 GB DatabaseApplication Database

Applications use standard SQL client libraries: ODBC, ADO.Net, PHP, … Application Internet Azure Cloud TDS (tcp) Security Boundary Load balancer forwards „sticky‟ sessions to TDS LB protocol tier TDS (tcp) Gateway Gateway Gateway Gateway Gateway Gateway Gateway: TDS protocol gateway, enforces AUTHN/AUTHZ policy; proxy to CloudDB TDS (tcp)L SQL SQL SQL SQL Scalability and Availability: Fabric, Failover, Replication, and Load balancing

Gateway Process AdminSvc TDS Endpoint Provisioning Endpoint Endpoint Protocol Parser Business Logic Services Connection MgmtL SQL SQL SQL SQL Scalability and Availability: Fabric, Failover, Replication, and Load balancing

Live DNS Cluster Customer Browser Live DNS Svc 1 Datacenter (Sub-Region) 5 Portal LB Gateway LB 2 4 3 6Front-end Node Front-end Node Front-end Node Front-end NodeAdmin Portal Admin Portal Gateway Gateway 7 Backend Node Backend Node Backend Node SQL Server SQL Server SQL Server Mgmt. Mgmt. Mgmt. Services Fabric Services Fabric Services Fabric

1 TDS Gateway Front-end Node TDS Session Protocol Parser 2 3 Gateway Logic Master Cluster Master Node 4 Master Node Components 7 5 8 6Backend Node 1 Backend Node 2 Backend Node 3 SQL Instance SQL Instance SQL Instance SQL DB SQL DB SQL DB Scalability and and Availability: Fabric,Failover,Replication, and Load balancing Scalability Availability: Fabric, Failover, Replication, and Load balancing

7 1 TDS Gateway Front-end Node TDS Session Protocol Parser 6 2 Gateway Logic Global Partition Map Master Node 8 3 Master Node Components 4 5Backend Node 1 Backend Node 2 Backend Node 3 SQL Instance SQL Instance SQL Instance SQL DB SQL DB SQL DB Scalability and and Availability: Fabric,Failover,Replication, and Load balancing Scalability Availability: Fabric, Failover, Replication, and Load balancing

Web Role Reporting

On Premises Windows Azure Local Agent Data Sync Service Sync Orchestrator Sync Orchestrator HTTPS SQL Server SQL Server SQL ServerSync Provider Proxy Provider Provider TDS SQL SQL Server Azure

SQL Azure Now Database Data Sync Service For Remote Offices CTP2 – End of CY10Retail Stores SQL Azure Sync On-Premises (Headquarters)

Data SyncWeb Role Reporting

Data SyncWeb Role Data Sync Reporting

www.microsoft.com/sqlazure

Service Bus Access Control ? SQL AzureApplications Windows Azure AppFabricWindows Azure Applications Windows Others

Service Bus 3) Discover Service Bus endpoint 1) Register service Registry endpoint 2) Expose Service Bus endpoint 5) Invoke operation 4) Invoke operation on service endpoint on Service Bus endpoint Client WCF Service (On-premises (On-premises) or cloud) Service BusAccess Control

Blobs SQL Azure On-Premises or Hosted Application

TablesWeb RoleInstance Users

Tables Queues Blobs Web Role Worker Role Instance Instance Users

SQL AzureWeb RoleInstance Users

SQL AzureWeb Role Service BusInstance On-premises Database Users

Queues Blobs Worker RoleWeb Role InstanceInstance User

http://comunidadwindows.orghttp://www.slideshare.net/ecastromhttp://ecastrom.blogspot.com http://go.microsoft.com/fwlink/?LinkId=158011 http://go.microsoft.com/?linkid=9682907

http://www.microsoft.com/sqlazurehttp://blogs.msdn.com/sqlazurehttp://msdn.microsoft.com/windowsazure/sqlazurehttp://msdn.microsoft.com/data

Dr. Eduardo Castro MartínezMicrosoft MVPecastro@mswindowscr.orghttp://comunidadwindows.orghttp://ecastrom.blogspot.com

Introduction to SQL Server Cloud Storage Azure

by Eduardo Castro on Mar 16, 2011

Accessibility

Categories

Upload Details

Usage Rights

Statistics

Introduction to SQL Server Cloud Storage Azure — Presentation Transcript