Microsoft Database Options
Upcoming SlideShare
Loading in...5
×
 

Microsoft Database Options

on

  • 2,173 views

Joe Shirey's deck delivered at the 2009Q4 Microsoft Architect Council events

Joe Shirey's deck delivered at the 2009Q4 Microsoft Architect Council events

Statistics

Views

Total Views
2,173
Views on SlideShare
2,165
Embed Views
8

Actions

Likes
0
Downloads
24
Comments
0

2 Embeds 8

http://blogs.msdn.com 6
http://www.slideshare.net 2

Accessibility

Categories

Upload Details

Uploaded via as Microsoft PowerPoint

Usage Rights

© All Rights Reserved

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Processing…
Post Comment
Edit your comment

Microsoft Database Options Microsoft Database Options Presentation Transcript

  • Data Options in the Cloud
    Joe Shirey
    joe.shirey@microsoft.com
  • Database Choices
    Value Props:
    Full h/w control – size/scale
    100% of API surface area
    Roll-your-own HA/DR/scale
    Value Props:
    100% of API surface area
    Roll-your-own HA/DR/scale
    Dedicated
    On-premise
    SQL Server or other s/w on-premise
    Resource governance @ machine
    Security @ DB Server/OS
    Value Props:
    Auto HA, Fault-Tolerance
    Friction-free scale
    Self-provisioning
    Subset of API surface area
    Resources
    Hosted
    Hosted SQL Server
    Resource governance @ VM
    Security @ DB Server/OS
    SQL Azure Database
    Virtual DB server
    Resource governance @ DB
    Security @ DB/Virtual Server
    Shared
    Low
    “Friction”/Control
    High
    SQL Azure V1 targets scenarios that live in the lower left quadrant
  • SQL Server 2008 R2
    Focus Areas
    Improvements in Scalability and Performance
    Enhanced Manageability
    Self Service Business Intelligence
    Master Data Management
  • Master Data Management
    Single source of truth for Master Data
    Thin client management
    Safeguards of data integrity
    Data versioning
    Import/Export capabilities
  • SQL Server Versions
    New with SQL 2008 R2
    • Datacenter Edition (256 logical processor support)
    • Parallel Data Warehouse Edition (100+ TB support)
  • Cloud Development
    WEB & CLOUDS
    third party cloud
    web applications
    developer experience
    use existing skills and tools
    compute
    storage
    management
    management
    relational data
    connectivity
    access control
    ON-PREMISES
    lob Applications
    composite applications
  • Storage in the Cloud
    WEB & CLOUDS
    third party cloud
    web applications
    developer experience
    use existing skills and tools
    management
    management
    connectivity
    access control
    compute
    relational data
    storage
    ON-PREMISES
    lob Applications
    composite applications
  • Why Data in the Cloud
    Capital Costs
    DBA (or lack thereof)
    High Availability
    Performance
    Scalability
  • Windows Azure Storage
    The goal is to allow users and services
    Anywhere at anytime access
    Store data for any length of time
    Scale to store any amount of data
    Be confident that the data will not be lost
    Pay for only what they use/store
  • Windows Azure Storage
    Storage
    Durable
    Scalable (capacity and throughput)
    Highly available
    Rich storage concepts
    Large user data items: blobs
    Service state: tables
    Service communication: queues
    Simple and familiar programming interfaces
    REST (HTTP and HTTPS)
    .NET accessible
  • Fundamental data abstractions
    Blobs – provide a simple interface for storing named files along with metadata for the file
    Tables – provide structured storage. a table is a set of entities, which contain a set of properties
    Queues – provide reliable storage and delivery of messages for an application
  • Blob Storage Concepts
    blob
    container
    account
    IMG001.JPG
    pictures
    MOV1.AVI
    IMG002.JPG
    sally
    movies
  • Blob Features and Functions
    Store large objects (up to 200GB)
    Associate metadata with blob
    metadata is <name, value> pairs, up to 8KB per blob
    set/get with or separate from blob data bits
    Standard REST Interface
    PutBlob
    Inserts a new blob or overwrites the existing blob
    GetBlob
    Get whole blob or a specific range
    DeleteBlob
  • Windows Azure Tables
    Provides structured storage
    Massively scalable tables
    Billions of entities (rows) and TBs of data
    Automatically scales across servers as traffic grows
    Highly available
    Anywhere at anytime access to your data
    Durable
    Data is replicated at least 3 times
    Familiar and easy to use programming interfaces
    ADO.NET data services – .NET 3.5 SP1
    .NET classes and LINQ
    REST - with any platform or language
  • Table Storage Concepts
    entity
    table
    account
    Name =…
    Email = …
    users
    Name =…
    Email = …
    sally
    Photo ID =…
    Date =…
    photo index
    Photo ID =…
    Date =…
  • Table Data Model
    Table
    A storage account can create many tables
    Table name is scoped by account
    Data is stored in tables
    A table is a set of entities (rows)
    An entity is a set of properties (columns)
    Entity
    Two “key” properties that together are the unique ID of the entity in the table
    PartitionKey – enables scalability
    RowKey – uniquely identifies the entity within the partition
  • Partition Deyand Partitions
    Every table has a partition key
    It is the first property (column) of your table
    Used to group entities in the table into partitions
    A table partition
    All entities in a table with the same partition key value
    Partition key is exposed in the programming model
    Allows application to control the granularity of the partitions and enable scalability
  • Partition Example
    Table partition – all entities in tablewith same partition key value
    Application controls granularity of partition
    Partition 1
    Partition 2
  • Purpose of the Partition Key
    Entity locality
    Entities in the same partition will be stored together
    Efficient querying and cache locality
    Entity group transactions
    Atomically perform multiple insert/update/delete over entities in same partition in a single transaction
    Table scalability
    We monitor the usage patterns of partitions
    Automatically load balance partitions
    Each partition can be served by a different storage node
    Scale to meet the traffic needs of your table
  • Choosing a Partition Key
    Granularity of entity group transactions
    Make the partition key only as big as you need it for entity group transactions
    Spread out load across partitions
    More partitions – makes it easier to automatically balance load
    Currently have one primary index
    Important to use a partition key that is common in your queries
    If partition key is part of query
    Fast access to retrieve entities within a single partition
    If partition key is not specified in a query
    Then every partition has to be scanned
  • Table Entities and Properties
    Each entity can have up to 255 properties
    Mandatory properties for every entity in table
    Partition key
    Row key
    All entities have a system maintained version
    No fixed schema for rest of properties
    Each property is stored as a <name, typed value> pair
    No schema stored for a table
    2 entities within the same table can have different properties
    Properties can be the standard .NET types
    String, binary, bool, DateTime, GUID, int, int64, and double
  • Table Programming Model
    Provide familiar and easy to use interfaces
    Leverage your .NET expertise
    Table entities are accessed as objects via ADO.NET Data Services – .NET 3.5 SP1
    LINQ – language Integrated query
    RESTful access to table and entities
    Insert/update/delete entities over the table
    Query over tables
    Get back a list of structured entities
  • Web + Worker Role Pattern
    Web role
    Web farm that handles request from the internet
    Push work items onto storage queue
    Worker role
    Process work item off storage queue
    Public internet
    n
    m
    Web role
    Worker role
    Q
    Load
    balancer
    Cloud storage (tables, blobs, queues)
  • Windows Azure Queues
    Provide reliable message delivery
    Simple, asynchronous work dispatch
    Programming semantics ensure that a message can be processed at least once
    Queues are highly available, durable and performance efficient
    Access is provided via REST
  • Queue Storage Concepts
    Message
    Queue
    Account
    128x128, http://…
    thumbnail jobs
    256x256, http://…
    sally
    http://…
    photo processing jobs
    http://…
  • Account, Queues and Messages
    An account can create many queues
    Queue name is scoped by the account
    Aqueue contains messages
    No limit on number of messages stored in a queue
    A message is stored for at most a week in a queue
    http://<Account>.queue.core.windows.net/<QueueName>
    Messages
    Message size <= 8 KB
    To store larger data, store data in blob/entity storage, and the blob/entity name in the message
  • queue programming API
    queues
    create/delete/clear queues
    inspect queue length
    messages
    enqueue (queuename, message)
    dequeue (queueName, invisibility time T)
    returns the message with a messageID
    makes the message invisible for time T
    delete(queuename, messageID)
  • Queue Best Practices
    Make message processing idempotent
    Need to deal with failures
    No fixed order for dequeue messages
    Invisible messages result in out of order processing
    Use the queue length to scale your workers
  • Demowindows azure storage
  • SQL Azure
    SQL Server Data Services false start
    Familiar relational model
    Uses existing APIs & tools
    Friction free provisioning and reduced management
    Built for the cloud with availability and scale
    Clear feedback: “I want a SQL database in the cloud”
  • Service Provisioning Model
    Each account has zero or more servers
    Azure wide, provisioned in a common portal
    Billing instrument
    Each server has one or more databases
    Contains metadata about the databases and usage
    Unit of authentication
    Unit of geo-location
    Generated DNS based name
    Each database has standard SQL objects
    Unit of consistency
    Unit of multi-tenancy
    Contains users, tables, views, indices, etc.
    Most granular unit of billing
    SKU’s
    Web edition -1 GB
    Business edition – 10 GB
    account
    server
    database
  • Architecture
    Shared infrastructure at SQL database and below
    Request routing, security and isolation
    Scalable HA technology provides the glue
    Automatic replication and failover
    Provisioning, metering and billing infrastructure
    Machine 5
    Machine 6
    Machine 4
    SQL Instance
    SQL Instance
    SQL Instance
    SQL DB
    SQL DB
    SQL DB
    UserDB1
    UserDB2
    UserDB3
    UserDB4
    UserDB1
    UserDB2
    UserDB3
    UserDB4
    UserDB1
    UserDB2
    UserDB3
    UserDB4
    SDS Provisioning (databases, accounts, roles, …, Metering, and Billing
    Scalability and Availability: Fabric, Failover, Replication, and Load balancing
    Scalability and Availability: Fabric, Failover, Replication, and Load balancing
  • Sample of SQL compatibility
    In Scope for V1
    Tables, indexes and views
    Stored Procedures
    Triggers
    Constraints
    Table variables, session temp tables (#t)
    …..
    Out of Scope for V1
    • Distributed Transactions
    • Distributed Query
    • CLR
    • Service Broker
    • Spatial
    • Physical server or catalog DDL and views
  • Logical vs. Physical Administration
    Customer can focus on logical administration
    Schema creation and management
    Query optimization
    Security management (logins, users, roles)
    Service handles physical management
    Automatically replicated with HA “out of the box”
    Transparent failover in case of failure
    Load balancing of data to ensure SLA
    DBA role places more focus on logical management
  • Programming Model
    Small data sets
    Use a single database
    Same model as on-premise SQL Server
    Large data sets
    Partition data across many databases
    Use parallel fan-out queries to fetch the data
    Application code must be partition aware
  • Feature Set Comparison
  • Management Tooling
    SQL Server 2008 R2 Management Studio
    SQLCMD
    Third party/ communityadd-ons
  • Data Migration
    BCP
    SSIS
    SQL Azure Migration Wizard
  • Customer Learning’s from TAP program
    Use SQL Azure to store metadata and BLOB storage for large files
    Highly elastic load patterns in a cost effective way is an industry challenge
    Combination of different SKUs
  • Sync Framework
    SQL Azure
    Local SQL DB
    Sync Process
    Sync Process
    Local Computer
    MSDN - http://msdn.microsoft.com/en-us/sync/default.aspx
  • DemoSQL Azure storage and Sync
  • Pricing
    Windows Azure Storage
    Bandwidth: $0.10 in / $0.15 out / GB
    $0.15/GB stored/month
    SQL Azure Storage
    Bandwidth: $0.10 in / $0.15 out / GB
    Web edition (1GB): $9.99/month
    Business edition (10GB): $99.99/month
  • Data Options in the Cloud
    joe.shirey@microsoft.com
    http://www.joeshirey.com