Microsoft Database Options

Data Options in the Cloud
Joe Shirey
joe.shirey@microsoft.com

Database Choices
Value Props:
Full h/w control – size/scale
100% of API surface area
Roll-your-own HA/DR/scale
Value Props:
100% of API surface area
Roll-your-own HA/DR/scale
Dedicated
On-premise
SQL Server or other s/w on-premise
Resource governance @ machine
Security @ DB Server/OS
Value Props:
Auto HA, Fault-Tolerance
Friction-free scale
Self-provisioning
Subset of API surface area
Resources
Hosted
Hosted SQL Server
Resource governance @ VM
Security @ DB Server/OS
SQL Azure Database
Virtual DB server
Resource governance @ DB
Security @ DB/Virtual Server
Shared
Low
“Friction”/Control
High
SQL Azure V1 targets scenarios that live in the lower left quadrant

SQL Server 2008 R2
Focus Areas
Improvements in Scalability and Performance
Enhanced Manageability
Self Service Business Intelligence
Master Data Management

Master Data Management
Single source of truth for Master Data
Thin client management
Safeguards of data integrity
Data versioning
Import/Export capabilities

SQL Server Versions
New with SQL 2008 R2
Datacenter Edition (256 logical processor support)
Parallel Data Warehouse Edition (100+ TB support)

Cloud Development
WEB & CLOUDS
third party cloud
web applications
developer experience
use existing skills and tools
compute
storage
management
management
relational data
connectivity
access control
ON-PREMISES
lob Applications
composite applications

Storage in the Cloud
WEB & CLOUDS
third party cloud
web applications
developer experience
use existing skills and tools
management
management
connectivity
access control
compute
relational data
storage
ON-PREMISES
lob Applications
composite applications

Why Data in the Cloud
Capital Costs
DBA (or lack thereof)
High Availability
Performance
Scalability

Windows Azure Storage
The goal is to allow users and services
Anywhere at anytime access
Store data for any length of time
Scale to store any amount of data
Be confident that the data will not be lost
Pay for only what they use/store

Storage
Durable
Scalable (capacity and throughput)
Highly available
Rich storage concepts
Large user data items: blobs
Service state: tables
Service communication: queues
Simple and familiar programming interfaces
REST (HTTP and HTTPS)
.NET accessible

Fundamental data abstractions
Blobs – provide a simple interface for storing named files along with metadata for the file
Tables – provide structured storage. a table is a set of entities, which contain a set of properties
Queues – provide reliable storage and delivery of messages for an application

Blob Storage Concepts
blob
container
account
IMG001.JPG
pictures
MOV1.AVI
IMG002.JPG
sally
movies

Blob Features and Functions
Store large objects (up to 200GB)
Associate metadata with blob
metadata is <name, value> pairs, up to 8KB per blob
set/get with or separate from blob data bits
Standard REST Interface
PutBlob
Inserts a new blob or overwrites the existing blob
GetBlob
Get whole blob or a specific range
DeleteBlob

Windows Azure Tables
Provides structured storage
Massively scalable tables
Billions of entities (rows) and TBs of data
Automatically scales across servers as traffic grows
Highly available
Anywhere at anytime access to your data
Durable
Data is replicated at least 3 times
Familiar and easy to use programming interfaces
ADO.NET data services – .NET 3.5 SP1
.NET classes and LINQ
REST - with any platform or language

Table Storage Concepts
entity
table
account
Name =…
Email = …
users
Name =…
Email = …
sally
Photo ID =…
Date =…
photo index
Photo ID =…
Date =…

Table Data Model
Table
A storage account can create many tables
Table name is scoped by account
Data is stored in tables
A table is a set of entities (rows)
An entity is a set of properties (columns)
Entity
Two “key” properties that together are the unique ID of the entity in the table
PartitionKey – enables scalability
RowKey – uniquely identifies the entity within the partition

Partition Deyand Partitions
Every table has a partition key
It is the first property (column) of your table
Used to group entities in the table into partitions
A table partition
All entities in a table with the same partition key value
Partition key is exposed in the programming model
Allows application to control the granularity of the partitions and enable scalability

Partition Example
Table partition – all entities in tablewith same partition key value
Application controls granularity of partition
Partition 1
Partition 2

Purpose of the Partition Key
Entity locality
Entities in the same partition will be stored together
Efficient querying and cache locality
Entity group transactions
Atomically perform multiple insert/update/delete over entities in same partition in a single transaction
Table scalability
We monitor the usage patterns of partitions
Automatically load balance partitions
Each partition can be served by a different storage node
Scale to meet the traffic needs of your table

Choosing a Partition Key
Granularity of entity group transactions
Make the partition key only as big as you need it for entity group transactions
Spread out load across partitions
More partitions – makes it easier to automatically balance load
Currently have one primary index
Important to use a partition key that is common in your queries
If partition key is part of query
Fast access to retrieve entities within a single partition
If partition key is not specified in a query
Then every partition has to be scanned

Table Entities and Properties
Each entity can have up to 255 properties
Mandatory properties for every entity in table
Partition key
Row key
All entities have a system maintained version
No fixed schema for rest of properties
Each property is stored as a <name, typed value> pair
No schema stored for a table
2 entities within the same table can have different properties
Properties can be the standard .NET types
String, binary, bool, DateTime, GUID, int, int64, and double

Table Programming Model
Provide familiar and easy to use interfaces
Leverage your .NET expertise
Table entities are accessed as objects via ADO.NET Data Services – .NET 3.5 SP1
LINQ – language Integrated query
RESTful access to table and entities
Insert/update/delete entities over the table
Query over tables
Get back a list of structured entities

Web + Worker Role Pattern
Web role
Web farm that handles request from the internet
Push work items onto storage queue
Worker role
Process work item off storage queue
Public internet
n
m
Web role
Worker role
Q
Load
balancer
Cloud storage (tables, blobs, queues)

Windows Azure Queues
Provide reliable message delivery
Simple, asynchronous work dispatch
Programming semantics ensure that a message can be processed at least once
Queues are highly available, durable and performance efficient
Access is provided via REST

Queue Storage Concepts
Message
Queue
Account
128x128, http://…
thumbnail jobs
256x256, http://…
sally
http://…
photo processing jobs
http://…

Account, Queues and Messages
An account can create many queues
Queue name is scoped by the account
Aqueue contains messages
No limit on number of messages stored in a queue
A message is stored for at most a week in a queue
http://<Account>.queue.core.windows.net/<QueueName>
Messages
Message size <= 8 KB
To store larger data, store data in blob/entity storage, and the blob/entity name in the message

queue programming API
queues
create/delete/clear queues
inspect queue length
messages
enqueue (queuename, message)
dequeue (queueName, invisibility time T)
returns the message with a messageID
makes the message invisible for time T
delete(queuename, messageID)

Queue Best Practices
Make message processing idempotent
Need to deal with failures
No fixed order for dequeue messages
Invisible messages result in out of order processing
Use the queue length to scale your workers

Demowindows azure storage

SQL Azure
SQL Server Data Services false start
Familiar relational model
Uses existing APIs & tools
Friction free provisioning and reduced management
Built for the cloud with availability and scale
Clear feedback: “I want a SQL database in the cloud”

Service Provisioning Model
Each account has zero or more servers
Azure wide, provisioned in a common portal
Billing instrument
Each server has one or more databases
Contains metadata about the databases and usage
Unit of authentication
Unit of geo-location
Generated DNS based name
Each database has standard SQL objects
Unit of consistency
Unit of multi-tenancy
Contains users, tables, views, indices, etc.
Most granular unit of billing
SKU’s
Web edition -1 GB
Business edition – 10 GB
account
server
database

Architecture
Shared infrastructure at SQL database and below
Request routing, security and isolation
Scalable HA technology provides the glue
Automatic replication and failover
Provisioning, metering and billing infrastructure
Machine 5
Machine 6
Machine 4
SQL Instance
SQL Instance
SQL Instance
SQL DB
SQL DB
SQL DB
UserDB1
UserDB2
UserDB3
UserDB4
UserDB1
UserDB2
UserDB3
UserDB4
UserDB1
UserDB2
UserDB3
UserDB4
SDS Provisioning (databases, accounts, roles, …, Metering, and Billing
Scalability and Availability: Fabric, Failover, Replication, and Load balancing
Scalability and Availability: Fabric, Failover, Replication, and Load balancing

Sample of SQL compatibility
In Scope for V1
Tables, indexes and views
Stored Procedures
Triggers
Constraints
Table variables, session temp tables (#t)
…..
Out of Scope for V1
Distributed Transactions
Distributed Query
CLR
Service Broker
Spatial
Physical server or catalog DDL and views

Logical vs. Physical Administration
Customer can focus on logical administration
Schema creation and management
Query optimization
Security management (logins, users, roles)
Service handles physical management
Automatically replicated with HA “out of the box”
Transparent failover in case of failure
Load balancing of data to ensure SLA
DBA role places more focus on logical management

Programming Model
Small data sets
Use a single database
Same model as on-premise SQL Server
Large data sets
Partition data across many databases
Use parallel fan-out queries to fetch the data
Application code must be partition aware

Feature Set Comparison

Management Tooling
SQL Server 2008 R2 Management Studio
SQLCMD
Third party/ communityadd-ons

Data Migration
BCP
SSIS
SQL Azure Migration Wizard

Customer Learning’s from TAP program
Use SQL Azure to store metadata and BLOB storage for large files
Highly elastic load patterns in a cost effective way is an industry challenge
Combination of different SKUs

Sync Framework
SQL Azure
Local SQL DB
Sync Process
Sync Process
Local Computer
MSDN - http://msdn.microsoft.com/en-us/sync/default.aspx

DemoSQL Azure storage and Sync

Pricing
Bandwidth: $0.10 in / $0.15 out / GB
$0.15/GB stored/month
SQL Azure Storage
Bandwidth: $0.10 in / $0.15 out / GB
Web edition (1GB): $9.99/month
Business edition (10GB): $99.99/month

Data Options in the Cloud
joe.shirey@microsoft.com
http://www.joeshirey.com

Microsoft Database Options

by David Chou on Dec 04, 2009

Accessibility

Categories

Tags

Upload Details

Usage Rights

2 Embeds 6

Statistics

Microsoft Database Options — Presentation Transcript

http://blogs.msdn.com	4
http://www.slideshare.net	2