Your SlideShare is downloading. ×
0
Microsoft Database Options
Microsoft Database Options
Microsoft Database Options
Microsoft Database Options
Microsoft Database Options
Microsoft Database Options
Microsoft Database Options
Microsoft Database Options
Microsoft Database Options
Microsoft Database Options
Microsoft Database Options
Microsoft Database Options
Microsoft Database Options
Microsoft Database Options
Microsoft Database Options
Microsoft Database Options
Microsoft Database Options
Microsoft Database Options
Microsoft Database Options
Microsoft Database Options
Microsoft Database Options
Microsoft Database Options
Microsoft Database Options
Microsoft Database Options
Microsoft Database Options
Microsoft Database Options
Microsoft Database Options
Microsoft Database Options
Microsoft Database Options
Microsoft Database Options
Microsoft Database Options
Microsoft Database Options
Microsoft Database Options
Microsoft Database Options
Microsoft Database Options
Microsoft Database Options
Microsoft Database Options
Microsoft Database Options
Upcoming SlideShare
Loading in...5
×

Thanks for flagging this SlideShare!

Oops! An error has occurred.

×
Saving this for later? Get the SlideShare app to save on your phone or tablet. Read anywhere, anytime – even offline.
Text the download link to your phone
Standard text messaging rates apply

Microsoft Database Options

1,301

Published on

Joe Shirey's deck delivered at the 2009Q4 Microsoft Architect Council events

Joe Shirey's deck delivered at the 2009Q4 Microsoft Architect Council events

Published in: Technology
0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total Views
1,301
On Slideshare
0
From Embeds
0
Number of Embeds
1
Actions
Shares
0
Downloads
26
Comments
0
Likes
0
Embeds 0
No embeds

Report content
Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
No notes for slide

Transcript

  • 1. Data Options in the Cloud<br />Joe Shirey<br />joe.shirey@microsoft.com<br />
  • 2. Database Choices<br />Value Props:<br />Full h/w control – size/scale<br />100% of API surface area<br />Roll-your-own HA/DR/scale<br />Value Props:<br />100% of API surface area<br />Roll-your-own HA/DR/scale<br />Dedicated<br />On-premise<br /> SQL Server or other s/w on-premise<br /> Resource governance @ machine<br /> Security @ DB Server/OS<br />Value Props:<br />Auto HA, Fault-Tolerance<br />Friction-free scale<br />Self-provisioning<br />Subset of API surface area<br />Resources<br />Hosted<br /> Hosted SQL Server<br /> Resource governance @ VM<br /> Security @ DB Server/OS<br />SQL Azure Database<br /> Virtual DB server<br />Resource governance @ DB<br /> Security @ DB/Virtual Server<br />Shared<br />Low<br />“Friction”/Control<br />High<br />SQL Azure V1 targets scenarios that live in the lower left quadrant<br />
  • 3. SQL Server 2008 R2<br />Focus Areas<br />Improvements in Scalability and Performance<br />Enhanced Manageability<br />Self Service Business Intelligence<br />Master Data Management<br />
  • 4. Master Data Management<br />Single source of truth for Master Data<br />Thin client management<br />Safeguards of data integrity<br />Data versioning<br />Import/Export capabilities<br />
  • 5. SQL Server Versions<br />New with SQL 2008 R2<br /><ul><li>Datacenter Edition (256 logical processor support)
  • 6. Parallel Data Warehouse Edition (100+ TB support)</li></li></ul><li>Cloud Development<br />WEB & CLOUDS<br />third party cloud<br />web applications<br />developer experience<br />use existing skills and tools<br />compute<br />storage<br />management<br />management<br />relational data<br />connectivity<br />access control<br />ON-PREMISES<br />lob Applications<br />composite applications<br />
  • 7. Storage in the Cloud<br />WEB & CLOUDS<br />third party cloud<br />web applications<br />developer experience<br />use existing skills and tools<br />management<br />management<br />connectivity<br />access control<br />compute<br />relational data<br />storage<br />ON-PREMISES<br />lob Applications<br />composite applications<br />
  • 8. Why Data in the Cloud<br />Capital Costs<br />DBA (or lack thereof)<br />High Availability<br />Performance<br />Scalability <br />
  • 9. Windows Azure Storage<br />The goal is to allow users and services<br />Anywhere at anytime access<br />Store data for any length of time <br />Scale to store any amount of data<br />Be confident that the data will not be lost <br />Pay for only what they use/store<br />
  • 10. Windows Azure Storage<br />Storage<br />Durable<br />Scalable (capacity and throughput)<br />Highly available<br />Rich storage concepts<br />Large user data items: blobs<br />Service state: tables<br />Service communication: queues<br />Simple and familiar programming interfaces<br />REST (HTTP and HTTPS)<br />.NET accessible<br />
  • 11. Fundamental data abstractions<br />Blobs – provide a simple interface for storing named files along with metadata for the file<br />Tables – provide structured storage. a table is a set of entities, which contain a set of properties<br />Queues – provide reliable storage and delivery of messages for an application<br />
  • 12. Blob Storage Concepts<br />blob<br />container<br />account<br />IMG001.JPG<br />pictures<br />MOV1.AVI<br />IMG002.JPG<br />sally<br />movies<br />
  • 13. Blob Features and Functions<br />Store large objects (up to 200GB)<br />Associate metadata with blob<br />metadata is &lt;name, value&gt; pairs, up to 8KB per blob<br />set/get with or separate from blob data bits<br />Standard REST Interface<br />PutBlob<br />Inserts a new blob or overwrites the existing blob<br />GetBlob<br />Get whole blob or a specific range<br />DeleteBlob<br />
  • 14. Windows Azure Tables<br />Provides structured storage<br />Massively scalable tables<br />Billions of entities (rows) and TBs of data<br />Automatically scales across servers as traffic grows<br />Highly available<br />Anywhere at anytime access to your data<br />Durable<br />Data is replicated at least 3 times<br />Familiar and easy to use programming interfaces<br />ADO.NET data services – .NET 3.5 SP1<br />.NET classes and LINQ<br />REST - with any platform or language<br />
  • 15. Table Storage Concepts<br />entity<br />table<br />account<br />Name =…<br />Email = …<br />users<br />Name =…<br />Email = …<br />sally<br />Photo ID =…<br />Date =…<br />photo index<br />Photo ID =…<br />Date =…<br />
  • 16. Table Data Model<br />Table<br />A storage account can create many tables<br />Table name is scoped by account<br />Data is stored in tables<br />A table is a set of entities (rows)<br />An entity is a set of properties (columns)<br />Entity<br />Two “key” properties that together are the unique ID of the entity in the table<br />PartitionKey – enables scalability<br />RowKey – uniquely identifies the entity within the partition<br />
  • 17. Partition Deyand Partitions<br />Every table has a partition key<br />It is the first property (column) of your table<br />Used to group entities in the table into partitions<br />A table partition <br />All entities in a table with the same partition key value<br />Partition key is exposed in the programming model <br />Allows application to control the granularity of the partitions and enable scalability<br />
  • 18. Partition Example<br />Table partition – all entities in tablewith same partition key value<br />Application controls granularity of partition<br />Partition 1<br />Partition 2<br />
  • 19. Purpose of the Partition Key<br />Entity locality<br />Entities in the same partition will be stored together<br />Efficient querying and cache locality<br />Entity group transactions <br />Atomically perform multiple insert/update/delete over entities in same partition in a single transaction<br />Table scalability<br />We monitor the usage patterns of partitions<br />Automatically load balance partitions<br />Each partition can be served by a different storage node<br />Scale to meet the traffic needs of your table<br />
  • 20. Choosing a Partition Key<br />Granularity of entity group transactions<br />Make the partition key only as big as you need it for entity group transactions<br />Spread out load across partitions<br />More partitions – makes it easier to automatically balance load<br />Currently have one primary index<br />Important to use a partition key that is common in your queries<br />If partition key is part of query<br />Fast access to retrieve entities within a single partition<br />If partition key is not specified in a query <br />Then every partition has to be scanned<br />
  • 21. Table Entities and Properties<br />Each entity can have up to 255 properties<br />Mandatory properties for every entity in table<br />Partition key<br />Row key <br />All entities have a system maintained version<br />No fixed schema for rest of properties<br />Each property is stored as a &lt;name, typed value&gt; pair<br />No schema stored for a table<br />2 entities within the same table can have different properties<br />Properties can be the standard .NET types <br />String, binary, bool, DateTime, GUID, int, int64, and double<br />
  • 22. Table Programming Model<br />Provide familiar and easy to use interfaces<br />Leverage your .NET expertise<br />Table entities are accessed as objects via ADO.NET Data Services – .NET 3.5 SP1<br />LINQ – language Integrated query<br />RESTful access to table and entities<br />Insert/update/delete entities over the table<br />Query over tables<br />Get back a list of structured entities<br />
  • 23. Web + Worker Role Pattern<br />Web role<br />Web farm that handles request from the internet<br />Push work items onto storage queue<br />Worker role<br />Process work item off storage queue<br />Public internet<br />n<br />m<br />Web role<br />Worker role<br />Q<br />Load<br /> balancer<br />Cloud storage (tables, blobs, queues)<br />
  • 24. Windows Azure Queues<br />Provide reliable message delivery<br />Simple, asynchronous work dispatch<br />Programming semantics ensure that a message can be processed at least once<br />Queues are highly available, durable and performance efficient<br />Access is provided via REST<br />
  • 25. Queue Storage Concepts<br />Message<br />Queue<br />Account<br />128x128, http://…<br />thumbnail jobs<br />256x256, http://…<br />sally<br />http://…<br />photo processing jobs<br />http://…<br />
  • 26. Account, Queues and Messages<br />An account can create many queues<br />Queue name is scoped by the account<br />Aqueue contains messages<br />No limit on number of messages stored in a queue<br />A message is stored for at most a week in a queue<br />http://&lt;Account&gt;.queue.core.windows.net/&lt;QueueName&gt;<br />Messages<br />Message size &lt;= 8 KB<br />To store larger data, store data in blob/entity storage, and the blob/entity name in the message<br />
  • 27. queue programming API<br />queues<br />create/delete/clear queues<br />inspect queue length<br />messages<br />enqueue (queuename, message)<br />dequeue (queueName, invisibility time T)<br />returns the message with a messageID<br />makes the message invisible for time T<br />delete(queuename, messageID)<br />
  • 28. Queue Best Practices<br />Make message processing idempotent<br />Need to deal with failures<br />No fixed order for dequeue messages<br />Invisible messages result in out of order processing <br />Use the queue length to scale your workers<br />
  • 29. Demowindows azure storage<br />
  • 30. SQL Azure<br />SQL Server Data Services false start<br />Familiar relational model<br />Uses existing APIs & tools<br />Friction free provisioning and reduced management<br />Built for the cloud with availability and scale<br />Clear feedback: “I want a SQL database in the cloud”<br />
  • 31. Service Provisioning Model<br />Each account has zero or more servers<br />Azure wide, provisioned in a common portal<br />Billing instrument<br />Each server has one or more databases<br />Contains metadata about the databases and usage<br />Unit of authentication<br />Unit of geo-location<br />Generated DNS based name<br />Each database has standard SQL objects<br />Unit of consistency<br />Unit of multi-tenancy<br />Contains users, tables, views, indices, etc.<br />Most granular unit of billing<br />SKU’s <br />Web edition -1 GB<br />Business edition – 10 GB<br /> account<br /> server<br /> database<br />
  • 32. Architecture<br />Shared infrastructure at SQL database and below<br />Request routing, security and isolation<br />Scalable HA technology provides the glue<br />Automatic replication and failover<br />Provisioning, metering and billing infrastructure<br />Machine 5<br />Machine 6<br />Machine 4<br />SQL Instance<br />SQL Instance<br />SQL Instance<br />SQL DB<br />SQL DB<br />SQL DB<br />UserDB1<br />UserDB2<br />UserDB3<br />UserDB4<br />UserDB1<br />UserDB2<br />UserDB3<br />UserDB4<br />UserDB1<br />UserDB2<br />UserDB3<br />UserDB4<br />SDS Provisioning (databases, accounts, roles, …, Metering, and Billing<br />Scalability and Availability: Fabric, Failover, Replication, and Load balancing<br />Scalability and Availability: Fabric, Failover, Replication, and Load balancing<br />
  • 33. Sample of SQL compatibility<br />In Scope for V1<br />Tables, indexes and views<br />Stored Procedures<br />Triggers<br />Constraints<br />Table variables, session temp tables (#t)<br />…..<br />Out of Scope for V1<br /><ul><li>Distributed Transactions
  • 34. Distributed Query
  • 35. CLR
  • 36. Service Broker
  • 37. Spatial
  • 38. Physical server or catalog DDL and views</li></li></ul><li>Logical vs. Physical Administration<br />Customer can focus on logical administration<br />Schema creation and management<br />Query optimization<br />Security management (logins, users, roles)<br />Service handles physical management<br />Automatically replicated with HA “out of the box”<br />Transparent failover in case of failure<br />Load balancing of data to ensure SLA<br />DBA role places more focus on logical management<br />
  • 39. Programming Model<br />Small data sets<br />Use a single database<br />Same model as on-premise SQL Server<br />Large data sets<br />Partition data across many databases<br />Use parallel fan-out queries to fetch the data<br />Application code must be partition aware<br />
  • 40. Feature Set Comparison<br />
  • 41. Management Tooling<br />SQL Server 2008 R2 Management Studio<br />SQLCMD<br />Third party/ communityadd-ons<br />
  • 42. Data Migration <br />BCP<br />SSIS<br />SQL Azure Migration Wizard<br />
  • 43. Customer Learning’s from TAP program<br />Use SQL Azure to store metadata and BLOB storage for large files<br />Highly elastic load patterns in a cost effective way is an industry challenge<br />Combination of different SKUs<br />
  • 44. Sync Framework<br />SQL Azure<br />Local SQL DB<br />Sync Process<br />Sync Process<br />Local Computer<br />MSDN - http://msdn.microsoft.com/en-us/sync/default.aspx<br />
  • 45. DemoSQL Azure storage and Sync<br />
  • 46. Pricing<br />Windows Azure Storage<br />Bandwidth: $0.10 in / $0.15 out / GB<br />$0.15/GB stored/month<br />SQL Azure Storage<br />Bandwidth: $0.10 in / $0.15 out / GB<br />Web edition (1GB): $9.99/month<br />Business edition (10GB): $99.99/month<br />
  • 47. Data Options in the Cloud<br />joe.shirey@microsoft.com<br />http://www.joeshirey.com<br />

×