Building services using windows azure


Published on

Published in: Technology, Business
  • Be the first to comment

  • Be the first to like this

No Downloads
Total views
On SlideShare
From Embeds
Number of Embeds
Embeds 0
No embeds

No notes for slide

Building services using windows azure

  1. 1. Windows Azure Platform<br />Training Workshop<br />
  2. 2. Presentation Outline (hidden slide):<br />Technical Level: 300<br />Intended Audience: Developers<br />Objectives (what do you want the audience to take away):<br />1. To be able to build and deploy a Windows Azure Service<br />2. To be able to use blobs, tables and queues effectively<br />3. Understand how to use the API for logging<br />Presentation Outline (including demos):<br />Compute<br />Demo: Building and Deploying a Service<br />Demo: Logging and Configuration<br />Demo: Worker Roles<br />Storage<br />Demo: Using Blobs<br />Demo: Using Queues<br />Demo: Using Tables<br />Advanced Table Concepts<br />
  3. 3. Building Services using Windows Azure<br />Name<br />Title<br />Organization<br />Email<br />
  4. 4. Windows Azure<br />Windows Azure is the foundation of Microsoft’s Cloud Platform<br />It is an “Operating System in the Cloud” and provides Essential Services for the cloud<br />Virtualized Computation<br />Scalable Storage<br />Automatic Management<br />Developer SDK<br />
  5. 5. Role Programming Model<br />Inherits RoleEntryPoint<br />OnStart() Method<br />Called by Fabric on startup, allows you to perform initialization tasks.<br />Reports Busy status to load balancer until you return true.<br />OnStop() Method<br />Called when role is to be shutdown, graceful exit.<br />Run() Method<br />Main logic is here – can do anything, typically loop and never exit.<br />
  6. 6. Web Role<br />ASP.NET 3.5 SP1 – 64bit<br />IIS7 Hostable Web Core<br />Hosts<br />Webformsor MVC<br />FastCGI applications (e.g. PHP)<br />Http(s)/TCP<br />WebRole<br />New web.roleConfig to support FastCGI<br />Support same Role model semantics as Worker Role (e.g. OnStart, Run, etc.)<br />
  7. 7. Packaging & Deployment<br />ServicePackage<br />ServiceConfiguration<br />
  8. 8. Building and Deploying a Service<br />demo<br />
  9. 9. Configuration<br />Service Configuration<br />Serviceconfiguration.csdef– Service Model<br />ServiceConfiguration.cscfg – instance data<br />RoleEnvironment.GetConfigurationSettingValue()<br />Don’t use web.config for values you wish to change at runtime<br />Web.config change requires re-deploy<br />
  10. 10. Monitoring<br />No Debugging in Cloud<br />Instrument your application using Trace, Debug<br />DiagnosticMonitorTraceListener<br />Use Diagnostics API to Configure and Collect<br />Event Logs<br />Performance Counters<br />Trace/Debug information (logging)<br />IIS Logs, Failed Request Logs<br />Crash Dumps or Arbitrary files<br />Request data on demand or scheduled<br />Transferred into your table and/or blob storage<br />Everything is remotely configurable<br />
  11. 11. Logging and Configuration<br />demo<br />
  12. 12. Using Queues to Decouple Roles<br />Use Queue to Decouple<br />Design for Idempotency<br />multiple applications of the operation does not change the result<br />Web Role<br />Worker Role<br />Worker Role<br />Web Role<br />Worker Role<br />Worker Role<br />Storage<br />Queue<br />LB<br />LB<br />
  13. 13. Using the Worker Roles<br />demo<br />
  14. 14. Upgrading Your Application<br />Two Models: VIP Swap and In-Place Upgrade<br />VIP Swap:<br />Uses Staging and Production environments.<br />Allows to quickly swap environments.<br />Production: v1  Staging: v2, after swap then Production: v2  Staging: v1.<br />In-Place Upgrade<br />Performs a rolling upgrade on live service.<br />Entire service or a single role<br />Manual or Automatic across update domains<br />
  15. 15. Storage<br />Blobs – Provide a simple interface for storing named files along with metadata for the file <br />Drives – A durable NTFS volume backed by blob storage<br />Tables – Provide structured storage; A Table is a set of entities, which contain a set of properties<br />Queues – Provide reliable storage and delivery of messages for an application<br />
  16. 16. Storage in the Dev Fabric<br />Provides a local “Mock” storage<br />Emulates storage in cloud<br />Allows offline development<br />Requires SQL Express 2005/2008<br />
  17. 17. Storage in the cloud<br />Create a Storage Account<br />Endpoints<br />Access Keys<br />CDN<br />
  18. 18. Windows Azure Storage Account<br />User creates a globally unique storage account name<br />Can choose geo-location to host storage account<br />“US Anywhere”, “US North Central”, “US South Central”, <br />Can co-locate storage account with compute account<br />Receive a 256 bit secret key when creating account<br />Storage Account Capacity at Commercial Availability<br />Each storage account can store up to 100 TB <br />Default limit of 5 storage accounts per subscription <br />
  19. 19. Blob Containers<br />Number of Blob Containers<br />Can have has many Blob Containers that will fit within the storage account limit <br />Blob Container<br />A container holds a set of blobs<br />Set access policies at the container level <br />Private or Public accessible<br />Associate Metadata with Container<br />Metadata are <name, value> pairs<br />Up to 8KB per container<br />List the blobs in a container<br />Can now include Blob Metadata and MD5 (new)<br />
  20. 20. Blob Features and Functions<br />Store Large Objects (100s of GBs in size)<br />Associate Metadata with Blob<br />Metadata is <name, value> pairs, Up to 8KB per blob<br />Set/Get with or separate from blob data bits<br />Standard REST Interface<br />PutBlob<br />Inserts a new blob, overwrites the existing blob<br />GetBlob<br />Get whole blob or a specific range<br />DeleteBlob<br />CopyBlob(new)<br />SnapshotBlob(new)<br />LeaseBlob(new)<br />
  21. 21. Blocks or Pages<br />Blob<br />Container<br />Account<br />Blob Storage ConceptsKey concepts account, container, blob and blocks/pages<br />IMG001.JPG<br />Pictures<br />IMG002.JPG<br />Account<br />Block/Page 1<br />Movies<br />MOV1.AVI<br />Block/Page 2<br />Block /Page 3<br />
  22. 22. Two Types of Blobs Under the Hood<br />Block Blob <br />Targeted at streaming workloads<br />Each blob consists of a sequence of blocks<br />Each block is identified by a Block ID<br />Size limit 200GB per blob<br />Page Blob (new)<br />Targeted at random read/write workloads<br />Each blob consists of an array of pages<br />Each page is identified by its offset from the start of the blob<br />Size limit 1TB per blob<br />
  23. 23. Block Blob Details<br />Block can be up to 4MB each <br />Each block can be variable size<br />Each block has a 64 byte ID<br />Scoped by blob name and stored with the blob<br />Block operation<br />PutBlock<br />Puts an uncommitted block defined by the block ID for the blob <br />Block List Operations<br />PutBlockList<br />Provide the list of blocks to comprise the readable version of the blob<br />Can use blocks from uncommitted or committed list to update blob<br />GetBlockList<br />Returns the list of blocks, committed or uncommitted for a blob<br />Block ID and Size of Block is returned for each block<br />
  24. 24. Page Blob Details<br />Page Blob is created with a Max Blob Size<br />Can change the max size of the blob at anytime.<br />Address space is broken up into fixed sized 512 byte pages for updates<br />Page update operations – must be page aligned<br />PutPage- limited to 4MB<br />Overwrite range of pages starting at the specified offset<br />ClearPage- can specify up to max size of blob<br />Clear range of pages at the offset<br />Reading a Page Blob<br />GetBlob – can read from any byte offset for any valid range<br />What parts of the Page Blob have stored pages in them<br />GetPageRange<br />Get valid page ranges in the blob<br />Only charged for pages with data stored in them<br />
  25. 25. Choosing Between Block and Page Blob <br />Block Blob<br />Targeted at streaming workloads<br />Update semantics<br />Upload a bunch of blocks. Then commit change.<br />Concurrency: ETag Checks<br />Page Blob<br />Targeted at random read/write workloads<br />Update Semantics<br />Immediate update<br />Concurrency: Leases<br />
  26. 26. Using Blobs<br />demo<br />
  27. 27. Summary Of Windows Azure Blobs<br />Blob Access Patterns<br />Block Blobs – streaming<br />Page Blobs – random read/write (new)<br />New Blob Operations<br />Copy, Snapshot, and Lease work for both types<br />New Ways of Accessing and Serving Blob Content (see MSDN)<br />Content Delivery Network access<br />Custom Domain Names<br />Root Blob Container<br />Shared Access Signatures <br />Additional new features<br />ListBlob, GetBlob, Blob Properties (see MSDN docs)<br />
  28. 28. Queues<br />Simple asynchronous dispatch queue<br />Create and delete queues<br />Message:<br />Retrieved at least once<br />Max size 8kb<br />Operations:<br />put<br />get<br />delete<br />
  29. 29. Queue Storage ConceptsAccount, queue and message<br />Message<br />Queue<br />Account<br />128x128, http://…<br />Thumbnail Jobs<br />256x256, http://…<br />Account<br />http://…<br />Indexing Jobs<br />http://…<br />
  30. 30. Using Queues<br />demo<br />
  31. 31. Summary Of Queues<br />Provide reliable message delivery<br />Allows Messages to be retrieved and processed at least once<br />No limit on number of messages stored in a Queue<br />Message size is <=8KB<br />
  32. 32. Tables<br />Entities and properties (rows & columns)<br />Tables scoped by account<br />Designed for billions+<br />Scale-out using partitions<br />Partition key & row key<br />Operations performed on partitions<br />Efficient queries<br />No limit on number of partitions<br />Use ADO.NET Data Services<br />
  33. 33. Table Storage ConceptsAccount, table and entity<br />Entity<br />Table<br />Account<br />Name=…hash=…<br />Users<br />Name=…hash=…<br />Account<br />Tag=…id=…<br />PhotoIndex<br />Tag=…,id=…<br />
  34. 34. Entities and Properties<br />Each Entity can have up to 255 properties<br />Every Entity has fixed key properties<br />Partition key<br />Row key<br />Timestamp<br />No fixed schema for rest of properties<br />2 entities in the same table can have different properties<br />Properties stored as <Name, TypedValue> pairs<br />Each entity has a system maintained version<br />
  35. 35. Property Types<br />Partition key and Row key<br />String (up to 1KB)<br />Other properties<br />String (up to 64KB)<br />Binary (up to 64KB)<br />Bool<br />DateTime<br />GUID<br />Int<br />Int64<br />Double<br />
  36. 36. Partition Key And Partition<br />Every Entity has a partition key<br />All entities in a table with the same partition key value live in the same partition <br />Need to choose partitioning scheme to make data access scalable <br />
  37. 37. Partitioning Guidelines<br />Performance<br />Use a PartitionKey that is common in your queries<br />Always try to specify the partition key in the query<br />Entities with same partition key value are clustered<br />Batch capabilities and transaction are supported within a partition only.<br />Scalability<br />We monitor partition traffic<br />Automatically load balance partitions<br />Each partition can potentially be served by a different storage node<br />Scale to meet the traffic needs of your application<br />More partitions – makes it easier to balance load<br />
  38. 38. Using Tables<br />demo<br />
  39. 39. ClientB<br />Client A<br />Concurrent Updates<br />Version<br />Rating<br />1: Ch9, Jan-2, 4<br />1: Ch9, Jan-2, 5<br />2: Ch9, Jan-2, 5<br />Error: 412<br />5 : Ch9, Jan-1, 3<br />If-Match: 1<br />Ch9, Jan-2, 4<br />1 : Ch9, Jan-2, 2<br />1 : Ch9, Jan-2, 2<br />1 : Ch9, Jan-2, 2<br />2: Ch9, Jan-2, 5<br />If-Match: 1<br />Ch9, Jan-2, 5<br />9 : Ch9, Jan-3, 6<br />Use standard HTTP mechanisms – Etag and If-Match<br />Get entity – get system maintained version as ETag<br />Update Entities Locally – change rating<br />Send Update with version check - IF-Match with Etag<br />Success if version matches, and update version on Client-A<br />Precondition failed (412) if version does not match<br />
  40. 40. .NET: LINQ Take(N) function<br />Getting the Top N entities<br />serviceUri = new Uri("http://<account>");<br />DataServiceContextcontext = new DataServiceContext(serviceUri);<br />varallMessages = context.CreateQuery<Message>("Messages");<br />foreach (Message messagein allMessages.Take(100))<br />{<br />Console.WriteLine(message.Name);<br />}<br />REST: $top=N query string option<br />GET http://<serviceUri>/Messages?$top=100<br />
  41. 41. Pagination – Continuation Tokens<br />Send a request<br />GET http://<serviceUri>/Messages?$filter=...&$top=100<br />Get continuation token in response header<br />Messages<br />x-ms-continuation-NextPartitionKey: xxxxxxx<br />x-ms-continuation-NextRowKey: yyyyyy<br />Ch9, Date1,<br />100<br />Ch9, Date2,<br />Ch9, …<br />Set HTTP query parameters<br />Ch9,Date100,<br />GET http://<Uri>/Messages?$filter=...&$top=100<br />Ch9,Date101,<br />&NextPartitionKey=xxxxxxx<br />&NextRowKey=yyyyyy<br />Ch9, …<br />
  42. 42. Single Table Consistency<br />ACID transactions for single entity CUD<br />Insert/update/delete<br />Snapshot isolation for query within a single partition<br />Consistent view from start time of the query<br />No dirty (uncommitted) reads<br />Does not block concurrent updates<br />No snapshot isolation across partitions<br />No snapshot isolation across different continuations of a query<br />Batch transactions within partition only<br />
  43. 43. Cross Table Consistency<br />Application is responsible for maintaining consistency<br />Example<br />When a channel is deleted, delete all the messages for that channel<br />Failures can occur in the middle<br />Example - Application fails after deleting some messages<br />Use Windows Azure Queues to help ensure completion of operation<br />
  44. 44. Worker<br />Queue<br />Messages<br />Cross Table Consistency <br />Delete channel<br />Delete messages worker<br />Front end<br />Ch1, Msg1<br />Del Ch1<br />Del Ch5<br />Del Ch11<br />Del Ch1<br />2<br />Ch1, Msg2<br />Ch1, Msg3<br />Front End<br />Ch2, Msg1<br />Ch2, Msg2<br />Channels<br />Ch3, Msg1<br />Ch1,…<br />Ch2,…<br />1. Dequeue DelCh1<br />2. Delete Ch1 from Channels<br />3. Delete from Messages<br />4. Delete queue entry<br />
  45. 45. Delete channel<br />Delete messages worker<br />Front end<br />Worker 1<br />Del Ch1h<br />Queue<br />Messages<br />Resuming After Failure<br />Ch1, Msg1<br />Del Ch1<br />Del Ch5<br />Del Ch11<br />Del Ch1<br />2<br />Ch1, Msg2<br />Ch1, Msg3<br />Front End<br />Ch2, Msg1<br />Worker2<br />Ch2, Msg2<br />Channels<br />Ch3, Msg1<br />Ch1,…<br />Ch2,…<br />1. Dequeue DelCh1 and start delete<br />2. Fails after deleting Ch1 and Msg1<br />3. DelCh1 is visible again<br />4. Dequeue DelCh1 again<br />5. Repeat delete operations<br />
  46. 46. Table Summary<br />Windows Azure tables are<br />Massively Scalable<br />Highly Available<br />Simple familiar API<br />Use .NET –- ADO.NET Data Services and LINQ<br />Or use REST<br />Leverage your .NET expertise<br />
  47. 47. Learning Windows Azure<br /><br /><br />Download the SDK<br />You don’t need cloud access to develop!<br />Look at the samples in the SDK<br />Windows Azure Platform Training Kit<br />3 Windows Azure labs<br />Follow the team bloggers<br />
  48. 48. Q & A<br />
  49. 49. © 2008 Microsoft Corporation. All rights reserved. Microsoft, Windows, Windows Vista and other product names are or may be registered trademarks and/or trademarks in the U.S. and/or other countries.<br />The information herein is for informational purposes only and represents the current view of Microsoft Corporation as of the date of this presentation. Because Microsoft must respond to changing market conditions, it should not be interpreted to be a commitment on the part of Microsoft, and Microsoft cannot guarantee the accuracy of any information provided after the date of this presentation. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.<br />