Real World Patterns for Cloud Computing
•Download as PPTX, PDF•
0 likes•506 views
This document summarizes a presentation about leveraging cloud computing solutions on Microsoft Azure for storage, processing, and scaling of large amounts of digital content. It discusses using Azure blob storage to store content, queues to decouple processing instructions from the content, and worker roles to process content in a scalable manner. It also addresses challenges of storage account limitations and provides demos of round robin storage across multiple accounts and auto-scaling of worker roles to demonstrate elastic scaling in Azure.
Recommended
More Related Content
Similar to Real World Patterns for Cloud Computing
Similar to Real World Patterns for Cloud Computing (20)
More from Wade Wegner
More from Wade Wegner (8)
Recently uploaded
Recently uploaded (20)
Real World Patterns for Cloud Computing
- 2. Real-World Patterns for Cloud Computing Required Slide SESSION CODE: ARC304 Wade Wegner Technical Evangelist Microsoft Corporation Jerry Schulist Solutions Architect Tribune Company
- 3. What is the Tribune Company?
- 10. Three Key Scenarios 1. Storage 2. Compute 3. Scale Tools, Knowledge, Approach
- 11. What will we look at? 4 2 Image Processor (Queue) Content Ingestion Services 3 1 Image Processor(Worker Role) Content(Blob Storage)
- 12. So what?
- 13. Not just a case study
- 14. YOU! :
- 15. Three Key Scenarios 1. Storage 2. Compute 3. Scale
- 17. Things to consider Data management today Is it really infinite? Code-near and code-far scenarios Relational data?
- 18. Challenges 1. Running out of disk space 2. Handling archived digital content 3. Data stored across multiple datacenters
- 19. Blob storage Image Processor (Queue) Content Ingestion Services 1 Image Processor(Worker Role) Content(Blob Storage)
- 20. Leveraging Windows Azure Storage Uploading massive amounts of content into Windows Azure blobs. DEMO
- 21. Key Takeaways 1. Consolidate content in Windows Azure storage 2. Windows Azure Storage SDK best practices 3. Leverage the .NET Framework 4.0
- 22. Abstracting Work with Queues
- 23. Things to consider Loosely coupled and scalable solutions Reliable storage and delivery of messages Idempotency 8KBs per message; up to 100TBs
- 24. Challenges 1. Need to process content 2. Independent of upload 3. Blobs aren’t the best for writing instructions
- 25. Queues 2 Image Processor (Queue) Content Ingestion Services 1 Image Processor(Worker Role) Content(Blob Storage)
- 26. Leveraging Windows Azure Storage Using Queues to store instructions for content processing. DEMO
- 27. Key Takeaways 1. Use Windows Azure queues to decouple 2. Include all data relevant to work in message 3. Idempotent
- 28. Three Key Scenarios 1. Storage 2. Compute 3. Scale
- 29. Content Processing with Worker Roles
- 30. Things to consider Web Roles verses Worker Roles Decoupling with Windows Azure Queues
- 31. Challenges 1. Require multiple versions of content 2. JIT content processing is slow and expensive 3. Cost
- 32. Worker Role 2 Image Processor (Queue) Content Ingestion Services 3 1 Image Processor(Worker Role) Content(Blob Storage)
- 33. Leveraging Windows Azure Compute Processing content with Windows Azure Worker Roles DEMO
- 34. Key Takeaways 1. Use worker roles for long running processes 2. Process ahead of time 3. Handle exceptions to prevent role recycling
- 35. Three Key Scenarios 1. Storage 2. Compute 3. Scale
- 36. Scaling Storage Accounts
- 37. Things to consider Isn’t scale automatic? What does elastic scale actually mean? Multiple storage accounts? Multiple queues?
- 38. Challenges 1. Storage account limitation of 100 Terabytes 2. Handle this challenge in a supportable fashion
- 39. Worker Role 4 2 Image Processor (Queue) Content Ingestion Services 3 1 Image Processor(Worker Role) Content(Blob Storage)
- 40. Scaling in Windows Azure A round robin approach to elastic scale in Windows Azure storage DEMO
- 41. Key Takeaways 1. Multiple storage accounts 2. Round robin through accounts 3. Only pay for what you use
- 42. Scaling Windows Azure roles
- 43. Things to consider Isn’t scale automatic? What ways exist to scale? Cost
- 44. Challenges 1. Determine the number of roles required 2. Auto-scale
- 45. Scaling in Windows Azure An approach to implementing auto-scale in Windows Azure DEMO
- 46. Key Takeaways 1. Determine how to scale – queue or performance 2. Scaling service independent of application 3. Pay for what you use; scale wisely
- 49. Next Steps 1. Consider the impact that cloud computing will have on you, your company, and your customers 2. Explore the three scenarios we presented and how they impact you 3. Take advantage of TechEd, product teams, and each other
- 50. Resources Required Slide Learning Sessions On-Demand & Community Microsoft Certification & Training Resources www.microsoft.com/teched www.microsoft.com/learning Resources for IT Professionals Resources for Developers http://microsoft.com/technet http://microsoft.com/msdn
- 51. Required Slide Complete an evaluation on CommNet and enter to win!
- 52. Sign up for Tech·Ed 2011 and save $500 starting June 8 – June 31st http://northamerica.msteched.com/registration You can also register at the North America 2011 kiosk located at registrationJoin us in Atlanta next year
- 53. © 2010 Microsoft Corporation. All rights reserved. Microsoft, Windows, Windows Vista and other product names are or may be registered trademarks and/or trademarks in the U.S. and/or other countries. The information herein is for informational purposes only and represents the current view of Microsoft Corporation as of the date of this presentation. Because Microsoft must respond to changing market conditions, it should not be interpreted to be a commitment on the part of Microsoft, and Microsoft cannot guarantee the accuracy of any information provided after the date of this presentation. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.
Editor's Notes
- Good morning! Welcome to ARC304, Real-World Patterns for Cloud Computing.My name is Wade Wegner, and I’m a technical evangelist with Microsoft. Joining me is Jerry Schulist, a solutions architect with the Tribune Company.Before we get started, I want to answer the questions what is the Tribune Company, and why is knowing about them relevant to our talk on real-world patterns for cloud computing.
- Wade:Coming from Chicago, when I hear the word Tribune I immediately think of the Chicago Tribune. However, the Tribune Company is more than just the Chicago Tribune – it includes 7 other newspapers, 23 television stations, and a host of radio stations and web sites.Jerry:That’s right. The Tribune company is a large company that in addition to traditional media (like newspapers, radio, and TV) handles over 6.1 billion page views, and is supported by over 32 data centers, over 4,000 servers, and 75,000 sqft of raised floor.Wade:So, suffice to say, it’s a pretty massive company that’s been around for a long time and entrenched in many traditional forms of media.
- And this is part of the problem, because for the past 150 years, traditional media such as newspapers …
- … radio, and television has been a dominant – if not ONLY– source of information for consumers. Yet, in only the last few years …
- … these traditional channels have struggled to keep up with the onslaught of digital and social media.
- Consider the US Airways plane that landed in the Hudson river. Where did this story break? Twitter and TwitPic
- So, how does the Tribune Company compete? To stay relevant, the Tribune Company has had to reinvent the ways it delivers information to consumers …
- … taking an early bet on cloud computing and the Windows Azure platformJerry: Our goal is to create a repository that will bring together editorial content and contextual advertising so that it can be leveraged throughout Tribune to deliver a more compelling user experience all while reducing cost.
- In this session we’ll focus on three scenarios that the Tribune Company faced when moving to the cloud, what worked well for them, what was challenging, and also some best practices.We’ll look at …How they leverage Windows Azure storage to store TBs and PBs of content.How they leverage Windows Azure compute for content processing.And challenges solved by scaling in the cloud.You will walk away from this session with tools, knowledge and approach that you can use in your own cloud projects.
- To accomplish this, we’re going to look at an existing architecture used by the Tribune Company today. We’re going to walk through the solution used to …Take the rich content the Tribune has, and deploy it into Windows Azure storage.Along the way, we’re going to write instructions to Windows Azure queues, so that …… content processing worker roles can later process the data we’ve uploaded.And finally, we’re going to highlight how to scale both the storage and compute pieces of the solution.
- In this talk, we want to show you exactly how to apply these patterns and scenarios in the real world.As we present, I’d like to challenge you to think about your own company and clients, and how you might leverage these patterns and techniques to save them money and create new efficiencies.
- And to alleviate your fears, this session is NOT simply a customer case study. We’re going to show you real stuff and how to solve real challenges.
- Lastly, there’s an important analogy to draw from the Tribune Company story. Just as the Tribune Company is reinventing itself to stay relevant in a new world of digital and social media, so too do we as Architects and Developers have to reinvent ourselves to stay relevant in a world of cloud computing.So ask yourself, what are YOU doing to stay relevant?
- With that, let’s talk about storage.
- Two interruptions:1. 2. .NET Framework 4.0
- Follow best practices for Windows Azure storage SDKChange app.config so that you can increase parallel internet connectionsbloblClient.ParallelOperationThreadCountIncrease the BlobRequestOptions timeout valueLeverage the .NET Framework 4.0TaskParallel librariesParallel.ForEachEven the TaskParallel libraries can be tweaked to control how it parallelizes work!--Maxconnections set to 12 *’s the number of processorsblobContainer.ParallelOperationThreadCount = some numberUsed for files greater than 32 MBsDefaults to the # of CPU’s on your machineIncrease the timeout value on the BlobRequestOptionsBest practice calculation # of KBs uploading divided by KB throughput per sec
- Questions:1. Bandwidth costs w/in the data center – none!
- GrzegorzGogolowicz – business strategy w/in MicrosoftJerry, you have a mock of this?
- These are transitional times for big media. Consider where the Tribune Company has come from …
- … and where they are going as they redefine themselves for a new day of digital and doAnd this transition is not only true for the Tribune Company, as it redefinies itself in a new day of digital and social media, but also for architects and developers that are exploring new ways of leveraging the cloud.