Your SlideShare is downloading. ×
0
Exploring Windows Azure Cloud Storage
Exploring Windows Azure Cloud Storage
Exploring Windows Azure Cloud Storage
Exploring Windows Azure Cloud Storage
Exploring Windows Azure Cloud Storage
Exploring Windows Azure Cloud Storage
Exploring Windows Azure Cloud Storage
Exploring Windows Azure Cloud Storage
Exploring Windows Azure Cloud Storage
Exploring Windows Azure Cloud Storage
Exploring Windows Azure Cloud Storage
Exploring Windows Azure Cloud Storage
Exploring Windows Azure Cloud Storage
Exploring Windows Azure Cloud Storage
Exploring Windows Azure Cloud Storage
Exploring Windows Azure Cloud Storage
Exploring Windows Azure Cloud Storage
Exploring Windows Azure Cloud Storage
Exploring Windows Azure Cloud Storage
Exploring Windows Azure Cloud Storage
Exploring Windows Azure Cloud Storage
Exploring Windows Azure Cloud Storage
Exploring Windows Azure Cloud Storage
Exploring Windows Azure Cloud Storage
Exploring Windows Azure Cloud Storage
Exploring Windows Azure Cloud Storage
Exploring Windows Azure Cloud Storage
Exploring Windows Azure Cloud Storage
Exploring Windows Azure Cloud Storage
Exploring Windows Azure Cloud Storage
Exploring Windows Azure Cloud Storage
Exploring Windows Azure Cloud Storage
Exploring Windows Azure Cloud Storage
Exploring Windows Azure Cloud Storage
Exploring Windows Azure Cloud Storage
Exploring Windows Azure Cloud Storage
Exploring Windows Azure Cloud Storage
Exploring Windows Azure Cloud Storage
Exploring Windows Azure Cloud Storage
Exploring Windows Azure Cloud Storage
Exploring Windows Azure Cloud Storage
Exploring Windows Azure Cloud Storage
Upcoming SlideShare
Loading in...5
×

Thanks for flagging this SlideShare!

Oops! An error has occurred.

×
Saving this for later? Get the SlideShare app to save on your phone or tablet. Read anywhere, anytime – even offline.
Text the download link to your phone
Standard text messaging rates apply

Exploring Windows Azure Cloud Storage

2,125

Published on

Published in: Technology
0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total Views
2,125
On Slideshare
0
From Embeds
0
Number of Embeds
0
Actions
Shares
0
Downloads
33
Comments
0
Likes
0
Embeds 0
No embeds

Report content
Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
No notes for slide
  • Why we need?Illustrate with traditional way, file server approach
  • C:\\WindowsAzurePlatformKit\\Demos\\WindowsAzureGuestBookUsingBlobs
  • Transcript

    • 1. K.MohamedFaizal<br />Lead Consultant @ NCS (P) Ltd.<br />kmdfaizal@gmail.com<br />http://faizal-comeacross.blogspot.com<br />AzureUG.SG 26th Jan 2011.<br />Exploring Windows Azure Cloud Storage<br />
    • 2. About Me<br />14<br />
    • 3. Agenda<br />Cloud Storage Account Overview<br />Tables<br />Blobs<br />Queues<br />Storage Viewer / Explorer Tools<br />
    • 4. Cloud Storage<br />Blobs<br />Queues<br />Tables<br />The types of Azure Storage<br /><ul><li>Blobs – A file system
    • 5. Tables – Hierarchical Data Tables
    • 6. Queues – Asynchronous Messaging
    • 7. Drives – NTFS file systems (Blob)</li></ul>Each item in storage is stored in triplicate.<br />
    • 8. Make a guess!<br />What is the maximum size of data, a storage account can hold?<br />1 GB<br />100 GB<br />1 TB<br />100 TB<br />Answer: d) 100 TB<br />http://blogs.msdn.com/b/windowsazurestorage/archive/2010/05/10/windows-azure-storage-abstractions-and-their-scalability-targets.aspx<br />
    • 9. Windows Azure and SQL Azure <br />
    • 10. Creating a Storage Account<br />
    • 11. With client library<br />Automatically referenced in a cloud project<br />Supplies a nice .NET API on top of the REST API<br />You will need the following:<br />Account Name - movieconversion<br />Account Shared Key – NOcqFlqUwJPLlm…<br />Endpoint - blob.core.windows.net<br />-or-<br />UseDevelopmentStorage=true<br />
    • 12. Table Storage<br />
    • 13. Windows Azure Tables<br />Provides Structured Storage<br />Massively Scalable Tables<br />Billions of entities (rows) and TBs of data<br />Can use thousands of servers as traffic grows<br />Highly Available & Durable<br />Data is replicated several times<br />Familiar and Easy to use API<br />ADO.NET Data Services – .NET 3.5 SP1<br />.NET classes and LINQ<br />REST – with any platform or language<br />10<br />
    • 14. Tables<br />Is not relational<br />Can Not-<br />Create foreign key relationships between tables.<br />Perform server side joins between tables.<br />Create custom indexes on the tables.<br />No server side Count(), for example.<br />ADO.NET Data Service<br />Creates and consumes data via REST-based API's<br />The Table Service REST API implements the WCF Data Services API.<br />
    • 15. Table Structure<br />Account<br />Table<br />Account: MovieData<br />Table Name: Movies<br />Table Name: Customers<br />Entity<br />Star Wars<br />Star Trek<br />Fan Boys<br />Brian H. Prince<br />Jason Argonaut<br />Bill Gates<br /><ul><li>Tables store entities.</li></li></ul><li>Table Storage Concepts<br />Entities<br />Tables<br />Accounts<br />Email =…<br />Name = …<br />Users<br />Email =…<br />Name = …<br />moviesonline<br />Genre =…<br />Title = …<br />Movies<br />Genre =…<br />Title = …<br />13<br />
    • 16. Table Data Model<br />Table<br />A storage account can create many tables<br />Table name is scoped by account<br />Set of entities (i.e. rows)<br />Entity<br />Set of properties (columns)<br />Required properties<br />PartitionKey, RowKey and Timestamp<br />14<br />
    • 17. Required Entity Properties<br />PartitionKey & RowKey<br />Uniquely identifies an entity<br />Defines the sort order<br />Use them to scale your application<br />Timestamp <br />Read only<br />Optimistic Concurrency<br />15<br />
    • 18. PartitionKey And Partitions<br />PartitionKey<br />Used to group entities in the table into partitions<br />A table partition <br />All entities with same partition key value<br />Unit of scale<br />Control entity locality<br />Row key provides uniqueness within a partition<br />16<br />
    • 19. Required Properties<br />All entities must have the following properties:<br />Timestamp<br />PartitionKey<br />RowKey<br />PartitionKey + RowKey = “primary key”<br />
    • 20. Partitions and Partition Ranges<br />Server A<br />Table = Movies<br />[Action - Comedy)<br />Server A<br />Table = Movies<br />Server B<br />Table = Movies<br />[Comedy- Western)<br />18<br />
    • 21. Table Operations<br />Table<br />Create<br />Query<br />Delete<br />Entities<br />Insert<br />Update <br />Merge – Partial Update<br />Replace – Update entire entity<br />Delete<br />Query<br />Entity Group Transaction (new)<br />
    • 22. Creating a Table<br />Tables can be created and destroyed in code.<br />Wire up your storage in the on_start method.<br />Use a CloudTableClient to access the table.<br />
    • 23. Entity: Inherit the Required Fields<br />By inheriting from TableServiceEntity we can easily store our entity in an Azure Table.<br />
    • 24. Create a Context Class<br />WCF Data Services needs a context class. <br />This represents your connection to the table.<br />
    • 25. Adding an Entity<br />Other operations are just like normal ADO.NET Data Services,<br />such as delete, update, etc.<br />
    • 26. Deleting an Entity<br />
    • 27. Key Selection: Things to Consider<br />Scalability<br />Distribute load as much as possible<br />PartitionKey is critical for scalability<br />Hot partitions can be load balanced<br />Query Efficiency & Speed<br />Avoid frequent large scans<br />Parallelize queries<br />25<br />
    • 28. Movie Listing – Solution 1<br />Why do I need multiple PartitionKeys?<br />Account name as Partition Key <br />Movie title as RowKey since movie names need to be sorted<br />Category as a separate property<br />Does this scale?<br />26<br />
    • 29. Movie Listing – Solution 1<br />Single partition - Entire table served by one server<br />All requests served by that single server<br />Does not scale<br />Client<br />Client<br />Request<br />Request<br />Request<br />Request<br />Server A<br />27<br />
    • 30. Movie Listing – Solution 2<br />All movies partitioned by category<br />Allows system to load balance hot partitions<br />Load distributed<br />Better than single partition<br />Server A<br />Client<br />Client<br />Request<br />Request<br />Request<br />Request<br />Request<br />Request<br />Request<br />Request<br />Server B<br />28<br />
    • 31. Key Selection: Case Study 2<br />Log every transaction into a table for diagnostics<br />Scale Write Intensive Scenario<br />Logs can be retrieved for a given time range<br />29<br />
    • 32. Logging - Solution 1<br />Timestamp as Partition Key <br />Looks like an obvious choice<br />It is not a single partition as time moves forward <br />Append only<br />Requests to single partition range<br />Load balancingdoesnot help<br />Server may throttle<br />Server A<br />Applications<br />Client<br />Server B<br />Request<br />Request<br />Request<br />Request<br />30<br />
    • 33. Logging Solution 2 - Distribute "Append Only”<br />Prefix timestamp such that load is distributed<br />Id of the node logging<br />Hash into N buckets<br />Write load is now distributed <br />Better throughput<br />To query logs in time range<br />Parallelize it across prefix values<br />Server A<br />Applications<br />Client<br />Server B<br />Request<br />Request<br />Request<br />Request<br />31<br />
    • 34. Table Storage Demo<br />demo<br />
    • 35. Overview Blob Storage<br />
    • 36. What is a BLOB<br />Binary Large OBject<br />
    • 37. BLOBs in Azure & Size<br />Each BLOB can be up to 1TB in size<br />You can have as many containers and BLOBs as you want.<br />Containers can be created/destroyed on the fly<br />http://<account>.blob.core.windows.net/<container>/<blobname><br />http://cohowinery.blob.core.windows.net/images/pic01.jpg<br />Account: MovieConversion<br />blobs<br />Container: Originals<br />Container: Completed<br /><ul><li>Job1.mpg
    • 38. Job2.mpg
    • 39. Header.png
    • 40. Job 9.mpg
    • 41. Index.docx
    • 42. Job8.pdf</li></li></ul><li>Blob Storage Concepts<br />Blob<br />Container<br />Account<br />PIC01.JPG<br />images<br />MOV1.AVI<br />PIC02.JPG<br />sally<br />movies<br />
    • 43. Windows Azure Storage Services<br />New Features<br />Blobs – Provide a simple interface for storing named files along with metadata for the file<br />New Type of Blob<br />Page Blob<br />New Blob Operations<br />Update Blob<br />Copy Blob<br />Snapshot Blob<br />Lease Blob<br />New Ways of Accessing and Serving Blob Content<br />Custom Domain Names<br />Root Blob Container<br />Content Delivery Network access<br />Shared Access Signatures <br />Drives – Provides durable NTFS volumes for Windows Azure applications to use (Feb 2010)<br />
    • 44. Overview Queue Storage<br />
    • 45. What is a queue? <br />Queues are used to store messages<br />They are FIFO, and one way<br />A queue name must be lower case name, and URL friendly<br />Message<br />Queue<br />Account<br /> customer ID order ID http://…<br />order processing<br />cohowinery<br /> customer ID order ID http://…<br />
    • 46. Account, Queues and Messages<br />An account can create many queues<br />Queue Name is scoped by the account<br />A Queue contains messages<br />No limit on number of messages stored in a queue<br />Set a limit for message expiration<br />Messages<br />Message size <= 8 KB<br />To store larger data, store data in blob/entity storage, and the blob/entity name in the message<br />Message now has dequeue count<br />40<br />
    • 47.
    • 48. What Next<br />Our next UG Meeting on 30th March 2011<br />Microsoft Office Level 21, Singapore<br />
    • 49. Storage Viewer / Explorer Tools<br />Azure Storage Explorer<br />http://azurestorageexplorer.codeplex.com/<br />Myazurestorage.com<br />https://myazurestorage.com/<br />Cloud Xplorer<br />http://clumsyleaf.com/products/cloudxplorer<br />Cerebrata Cloud Storage Studio<br />http://www.cerebrata.com/products/cloudstoragestudio/<br />

    ×