Your SlideShare is downloading. ×
0
Exploring Windows Azure Cloud Storage
Exploring Windows Azure Cloud Storage
Exploring Windows Azure Cloud Storage
Exploring Windows Azure Cloud Storage
Exploring Windows Azure Cloud Storage
Exploring Windows Azure Cloud Storage
Exploring Windows Azure Cloud Storage
Exploring Windows Azure Cloud Storage
Exploring Windows Azure Cloud Storage
Exploring Windows Azure Cloud Storage
Exploring Windows Azure Cloud Storage
Exploring Windows Azure Cloud Storage
Exploring Windows Azure Cloud Storage
Exploring Windows Azure Cloud Storage
Exploring Windows Azure Cloud Storage
Exploring Windows Azure Cloud Storage
Exploring Windows Azure Cloud Storage
Exploring Windows Azure Cloud Storage
Exploring Windows Azure Cloud Storage
Exploring Windows Azure Cloud Storage
Exploring Windows Azure Cloud Storage
Exploring Windows Azure Cloud Storage
Exploring Windows Azure Cloud Storage
Exploring Windows Azure Cloud Storage
Exploring Windows Azure Cloud Storage
Exploring Windows Azure Cloud Storage
Exploring Windows Azure Cloud Storage
Exploring Windows Azure Cloud Storage
Exploring Windows Azure Cloud Storage
Exploring Windows Azure Cloud Storage
Exploring Windows Azure Cloud Storage
Exploring Windows Azure Cloud Storage
Exploring Windows Azure Cloud Storage
Exploring Windows Azure Cloud Storage
Exploring Windows Azure Cloud Storage
Exploring Windows Azure Cloud Storage
Exploring Windows Azure Cloud Storage
Exploring Windows Azure Cloud Storage
Exploring Windows Azure Cloud Storage
Exploring Windows Azure Cloud Storage
Exploring Windows Azure Cloud Storage
Exploring Windows Azure Cloud Storage
Upcoming SlideShare
Loading in...5
×

Thanks for flagging this SlideShare!

Oops! An error has occurred.

×
Saving this for later? Get the SlideShare app to save on your phone or tablet. Read anywhere, anytime – even offline.
Text the download link to your phone
Standard text messaging rates apply

Exploring Windows Azure Cloud Storage

2,104

Published on

Published in: Technology
0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total Views
2,104
On Slideshare
0
From Embeds
0
Number of Embeds
0
Actions
Shares
0
Downloads
33
Comments
0
Likes
0
Embeds 0
No embeds

Report content
Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
No notes for slide
  • Why we need?Illustrate with traditional way, file server approach
  • C:\\WindowsAzurePlatformKit\\Demos\\WindowsAzureGuestBookUsingBlobs
  • Transcript

    • 1. K.MohamedFaizal
      Lead Consultant @ NCS (P) Ltd.
      kmdfaizal@gmail.com
      http://faizal-comeacross.blogspot.com
      AzureUG.SG 26th Jan 2011.
      Exploring Windows Azure Cloud Storage
    • 2. About Me
      14
    • 3. Agenda
      Cloud Storage Account Overview
      Tables
      Blobs
      Queues
      Storage Viewer / Explorer Tools
    • 4. Cloud Storage
      Blobs
      Queues
      Tables
      The types of Azure Storage
      • Blobs – A file system
      • 5. Tables – Hierarchical Data Tables
      • 6. Queues – Asynchronous Messaging
      • 7. Drives – NTFS file systems (Blob)
      Each item in storage is stored in triplicate.
    • 8. Make a guess!
      What is the maximum size of data, a storage account can hold?
      1 GB
      100 GB
      1 TB
      100 TB
      Answer: d) 100 TB
      http://blogs.msdn.com/b/windowsazurestorage/archive/2010/05/10/windows-azure-storage-abstractions-and-their-scalability-targets.aspx
    • 9. Windows Azure and SQL Azure
    • 10. Creating a Storage Account
    • 11. With client library
      Automatically referenced in a cloud project
      Supplies a nice .NET API on top of the REST API
      You will need the following:
      Account Name - movieconversion
      Account Shared Key – NOcqFlqUwJPLlm…
      Endpoint - blob.core.windows.net
      -or-
      UseDevelopmentStorage=true
    • 12. Table Storage
    • 13. Windows Azure Tables
      Provides Structured Storage
      Massively Scalable Tables
      Billions of entities (rows) and TBs of data
      Can use thousands of servers as traffic grows
      Highly Available & Durable
      Data is replicated several times
      Familiar and Easy to use API
      ADO.NET Data Services – .NET 3.5 SP1
      .NET classes and LINQ
      REST – with any platform or language
      10
    • 14. Tables
      Is not relational
      Can Not-
      Create foreign key relationships between tables.
      Perform server side joins between tables.
      Create custom indexes on the tables.
      No server side Count(), for example.
      ADO.NET Data Service
      Creates and consumes data via REST-based API's
      The Table Service REST API implements the WCF Data Services API.
    • 15. Table Structure
      Account
      Table
      Account: MovieData
      Table Name: Movies
      Table Name: Customers
      Entity
      Star Wars
      Star Trek
      Fan Boys
      Brian H. Prince
      Jason Argonaut
      Bill Gates
      • Tables store entities.
    • Table Storage Concepts
      Entities
      Tables
      Accounts
      Email =…
      Name = …
      Users
      Email =…
      Name = …
      moviesonline
      Genre =…
      Title = …
      Movies
      Genre =…
      Title = …
      13
    • 16. Table Data Model
      Table
      A storage account can create many tables
      Table name is scoped by account
      Set of entities (i.e. rows)
      Entity
      Set of properties (columns)
      Required properties
      PartitionKey, RowKey and Timestamp
      14
    • 17. Required Entity Properties
      PartitionKey & RowKey
      Uniquely identifies an entity
      Defines the sort order
      Use them to scale your application
      Timestamp
      Read only
      Optimistic Concurrency
      15
    • 18. PartitionKey And Partitions
      PartitionKey
      Used to group entities in the table into partitions
      A table partition
      All entities with same partition key value
      Unit of scale
      Control entity locality
      Row key provides uniqueness within a partition
      16
    • 19. Required Properties
      All entities must have the following properties:
      Timestamp
      PartitionKey
      RowKey
      PartitionKey + RowKey = “primary key”
    • 20. Partitions and Partition Ranges
      Server A
      Table = Movies
      [Action - Comedy)
      Server A
      Table = Movies
      Server B
      Table = Movies
      [Comedy- Western)
      18
    • 21. Table Operations
      Table
      Create
      Query
      Delete
      Entities
      Insert
      Update
      Merge – Partial Update
      Replace – Update entire entity
      Delete
      Query
      Entity Group Transaction (new)
    • 22. Creating a Table
      Tables can be created and destroyed in code.
      Wire up your storage in the on_start method.
      Use a CloudTableClient to access the table.
    • 23. Entity: Inherit the Required Fields
      By inheriting from TableServiceEntity we can easily store our entity in an Azure Table.
    • 24. Create a Context Class
      WCF Data Services needs a context class.
      This represents your connection to the table.
    • 25. Adding an Entity
      Other operations are just like normal ADO.NET Data Services,
      such as delete, update, etc.
    • 26. Deleting an Entity
    • 27. Key Selection: Things to Consider
      Scalability
      Distribute load as much as possible
      PartitionKey is critical for scalability
      Hot partitions can be load balanced
      Query Efficiency & Speed
      Avoid frequent large scans
      Parallelize queries
      25
    • 28. Movie Listing – Solution 1
      Why do I need multiple PartitionKeys?
      Account name as Partition Key
      Movie title as RowKey since movie names need to be sorted
      Category as a separate property
      Does this scale?
      26
    • 29. Movie Listing – Solution 1
      Single partition - Entire table served by one server
      All requests served by that single server
      Does not scale
      Client
      Client
      Request
      Request
      Request
      Request
      Server A
      27
    • 30. Movie Listing – Solution 2
      All movies partitioned by category
      Allows system to load balance hot partitions
      Load distributed
      Better than single partition
      Server A
      Client
      Client
      Request
      Request
      Request
      Request
      Request
      Request
      Request
      Request
      Server B
      28
    • 31. Key Selection: Case Study 2
      Log every transaction into a table for diagnostics
      Scale Write Intensive Scenario
      Logs can be retrieved for a given time range
      29
    • 32. Logging - Solution 1
      Timestamp as Partition Key
      Looks like an obvious choice
      It is not a single partition as time moves forward
      Append only
      Requests to single partition range
      Load balancingdoesnot help
      Server may throttle
      Server A
      Applications
      Client
      Server B
      Request
      Request
      Request
      Request
      30
    • 33. Logging Solution 2 - Distribute "Append Only”
      Prefix timestamp such that load is distributed
      Id of the node logging
      Hash into N buckets
      Write load is now distributed
      Better throughput
      To query logs in time range
      Parallelize it across prefix values
      Server A
      Applications
      Client
      Server B
      Request
      Request
      Request
      Request
      31
    • 34. Table Storage Demo
      demo
    • 35. Overview Blob Storage
    • 36. What is a BLOB
      Binary Large OBject
    • 37. BLOBs in Azure & Size
      Each BLOB can be up to 1TB in size
      You can have as many containers and BLOBs as you want.
      Containers can be created/destroyed on the fly
      http://<account>.blob.core.windows.net/<container>/<blobname>
      http://cohowinery.blob.core.windows.net/images/pic01.jpg
      Account: MovieConversion
      blobs
      Container: Originals
      Container: Completed
    • Blob Storage Concepts
      Blob
      Container
      Account
      PIC01.JPG
      images
      MOV1.AVI
      PIC02.JPG
      sally
      movies
    • 43. Windows Azure Storage Services
      New Features
      Blobs – Provide a simple interface for storing named files along with metadata for the file
      New Type of Blob
      Page Blob
      New Blob Operations
      Update Blob
      Copy Blob
      Snapshot Blob
      Lease Blob
      New Ways of Accessing and Serving Blob Content
      Custom Domain Names
      Root Blob Container
      Content Delivery Network access
      Shared Access Signatures
      Drives – Provides durable NTFS volumes for Windows Azure applications to use (Feb 2010)
    • 44. Overview Queue Storage
    • 45. What is a queue?
      Queues are used to store messages
      They are FIFO, and one way
      A queue name must be lower case name, and URL friendly
      Message
      Queue
      Account
      customer ID order ID http://…
      order processing
      cohowinery
      customer ID order ID http://…
    • 46. Account, Queues and Messages
      An account can create many queues
      Queue Name is scoped by the account
      A Queue contains messages
      No limit on number of messages stored in a queue
      Set a limit for message expiration
      Messages
      Message size <= 8 KB
      To store larger data, store data in blob/entity storage, and the blob/entity name in the message
      Message now has dequeue count
      40
    • 47.
    • 48. What Next
      Our next UG Meeting on 30th March 2011
      Microsoft Office Level 21, Singapore
    • 49. Storage Viewer / Explorer Tools
      Azure Storage Explorer
      http://azurestorageexplorer.codeplex.com/
      Myazurestorage.com
      https://myazurestorage.com/
      Cloud Xplorer
      http://clumsyleaf.com/products/cloudxplorer
      Cerebrata Cloud Storage Studio
      http://www.cerebrata.com/products/cloudstoragestudio/

    ×