How to scale up, out or down in Windows Azure

4,439 views
4,162 views

Published on

Published in: Technology

How to scale up, out or down in Windows Azure

  1. 1. HOW TO SCALE UP,OUT OR DOWN INWINDOWS AZUREJUAN DE ABREUVP, Sales Director
  2. 2. # CSwebinar
  3. 3. OUTLINE• SCALABILITY Achieving linear scale Scale Up vs. Scale Out in Windows Azure Choosing VM Sizes• CACHING Approaches to caching Cache storage• ELASTICITY Scale out, scale back Automation of scaling HOW TO SCALE UP, OUT OR DOWN IN WINDOWS AZURE - JUAN DE ABREU - #CSwebinar
  4. 4. A PRIMER ON SCALESCALABILITY IS THE ABILITY TO ADD CAPACITY TO A COMPUTINGSYSTEM TO ALLOW IT TO PROCESS MORE WORK HOW TO SCALE UP, OUT OR DOWN IN WINDOWS AZURE - JUAN DE ABREU - #CSwebinar
  5. 5. A PRIMER ON SCALABILITY• VERTICAL SCALE UP • HORIZONTAL SCALE OUT Add more resources to a single Adding additional computation units and computation unit i.e. Buy a bigger box having them act in concert Move a workload to a computation unit Splitting workload across multiple with more resources computation units e.g. Windows Azure Storage moving a partition. HOW TO SCALE UP, OUT OR DOWN IN WINDOWS AZURE - JUAN DE ABREU - #CSwebinar
  6. 6. VERTICAL VS. HORIZONTAL• FOR SMALL SCENARIOS SCALE UP IS CHEAPER Code ‘just works’• FOR LARGER SCENARIOS SCALE OUT ONLY SOLUTION Massive diseconomies of scale 1 x 64 Way Server >>>$$$ 64 x 1 Way Servers. Shared resource contention becomes a problem• SCALE OUT OFFERS PROMISE OF LINEAR, INFINITE SCALE HOW TO SCALE UP, OUT OR DOWN IN WINDOWS AZURE - JUAN DE ABREU - #CSwebinar
  7. 7. SCALABILITY != PERFORMANCEOFTEN YOU WILL SACRIFICE RAW SPEED FOR SCALABILITYFOR EXAMPLE; ASP.NET SESSION STATE HOW TO SCALE UP, OUT OR DOWN IN WINDOWS AZURE - JUAN DE ABREU - #CSwebinar
  8. 8. ACHIEVING LINEAR SCALE OUT• REDUCE OR ELIMINATE SHARED RESOURCES• MINIMIZE RELIANCE ON TRANSACTIONS OR TRANSACTIONAL TYPE BEHAVIOUR• HOMOGENOUS, STATELESS COMPUTATION NODES We can then use simple work distribution methods, Load balancers, queue distribution Less reliance on expensive hardware H/A HOW TO SCALE UP, OUT OR DOWN IN WINDOWS AZURE - JUAN DE ABREU - #CSwebinar
  9. 9. UNITS OF SCALECONSOLIDATION OF ROLES PROVIDES MORE REDUNDANCY FOR SAMECREATE AS MANY ROLES AS YOU NEED ‘KNOBS’ TO ADJUST SCALE Clean Up Role Web Site Role’ WCF Role Loss of an instance results in 50% capacity loss in web site. Cache Build Role HOW TO SCALE UP, OUT OR DOWN IN WINDOWS AZURE - JUAN DE ABREU - #CSwebinar
  10. 10. UNITS OF SCALECONSOLIDATION OF ROLES PROVIDES MORE REDUNDANCY FOR SAMECREATE AS MANY ROLES AS YOU NEED ‘KNOBS’ TO ADJUST SCALE Web Driven Role Loss of an instance results in 25% capacity loss in web site. Queue Drive Role HOW TO SCALE UP, OUT OR DOWN IN WINDOWS AZURE - JUAN DE ABREU - #CSwebinar
  11. 11. VM SIZE IN WINDOWS AZURE• WINDOWS AZURE Supports Various VM Sizes ~800mb/s NIC shared across machine Set in Service Definition (*.csdef). All instances of role will be equi-sized<WorkerRole name=“myRole" vmsize="ExtraLarge"> HOW TO SCALE UP, OUT OR DOWN IN WINDOWS AZURE - JUAN DE ABREU - #CSwebinar
  12. 12. Remember… IF IT DOESN’T RUN FASTER ON MULTIPLE CORES ON YOUR DESKTOP … IT’S NOT GOING TO RUN FASTER ON MULTIPLE CORES IN THE CLOUD! HOW TO SCALE UP, OUT OR DOWN IN WINDOWS AZURE - JUAN DE ABREU - #CSwebinar
  13. 13. CHOOSING YOUR VM SIZE• DON’T JUST THROW BIG VMS AT EVERY PROBLEM• SCALE OUT ARCHITECTURES HAVE NATURAL PARALLELISM• TEST VARIOUS CONFIGURATIONS UNDER LOAD• SOME SCENARIOS WILL BENEFIT FROM MORE CORES Where moving data >$ parallel overhead E.g. Video processing• STATEFUL SERVICES Database server requiring full network bandwidth HOW TO SCALE UP, OUT OR DOWN IN WINDOWS AZURE - JUAN DE ABREU - #CSwebinar
  14. 14. CACHING
  15. 15. CACHING• CACHING CAN IMPROVE BOTH PERFORMANCE AND SCALABILITY Moving data closer to the consumer (Web/Worker) improves performance. Reducing load on the hard to scale data tier HOW TO SCALE UP, OUT OR DOWN IN WINDOWS AZURE - JUAN DE ABREU - #CSwebinar
  16. 16. CACHING SCENARIO: WEBSITE UI IMAGES• WEBSITE UI IMAGES • GOAL: A BETTER UI Largely static data Serve content once Included in every page Avoid round trip unless content changes Minimize traffic over the wire Fewer storage transactions Lower load on web roles HOW TO SCALE UP, OUT OR DOWN IN WINDOWS AZURE - JUAN DE ABREU - #CSwebinar
  17. 17. CACHING SCENARIO: RSS FEEDS• REGULAR RSS FEED • GOAL: A BETTER RSS FEED Data delivered from database/storage Minimize traffic over the wire Large content payload >1mb Fewer storage transactions Data changes irregularly Less hits on database Cost determined by client voracity HOW TO SCALE UP, OUT OR DOWN IN WINDOWS AZURE - JUAN DE ABREU - #CSwebinar
  18. 18. CACHING STRATEGIES• CLIENT SIDE CACHING• STATIC CONTENT GENERATION HOW TO SCALE UP, OUT OR DOWN IN WINDOWS AZURE - JUAN DE ABREU - #CSwebinar
  19. 19. CLIENT SIDE CACHING HOW TO SCALE UP, OUT OR DOWN IN WINDOWS AZURE - JUAN DE ABREU - #CSwebinar
  20. 20. CLIENT CACHING - ETAGS• ETAG == SOFT CACHING Header added on HTTP Response ETag: “ABCDEFG” Client does conditional HTTP GET If-None-Match: “ABCDEFG” Returns content if ETag no longer matches Implemented natively by Windows Azure Storage Supports client side caching Also used for optimistic concurrency control HOW TO SCALE UP, OUT OR DOWN IN WINDOWS AZURE - JUAN DE ABREU - #CSwebinar
  21. 21. CLIENT CACHING - ETAGS• BENEFITS • PROBLEMS Prevents client downloading un- Still requires round trip to server necessary data May require execution of server side code Out of the box support for simple ‘static to re-create ETag before checking content’ scenarios. HOW TO SCALE UP, OUT OR DOWN IN WINDOWS AZURE - JUAN DE ABREU - #CSwebinar
  22. 22. CLIENT CACHING – CACHE-CONTROL• Cache-Control: max-age == Hard Caching Header added on HTTP Response Cache-Control: max-age=2592000 Client may cache file without further request for 30 days Client will not re-check on every request Very useful for static files header_logo.png Used to determine TTL on CDN edge nodes Set this on Blob using x-ms-blob-cache-control HOW TO SCALE UP, OUT OR DOWN IN WINDOWS AZURE - JUAN DE ABREU - #CSwebinar
  23. 23. STATIC CONTENT GENERATION HOW TO SCALE UP, OUT OR DOWN IN WINDOWS AZURE - JUAN DE ABREU - #CSwebinar
  24. 24. STATIC CONTENT GENERATION• GENERATE CONTENT PERIODICALLY IN WORKER ROLE Can spin up workers just for generation Generate as triggered asynchronous operation• CONTENT MAY BE Full pages Resources (CSS Sprites, PDF/XPS, Images etc.…) Content fragments• PUSH STATIC CONTENT INTO BLOB STORAGE Serve direct out of Blob storage May also be able to use persistent local storage HOW TO SCALE UP, OUT OR DOWN IN WINDOWS AZURE - JUAN DE ABREU - #CSwebinar
  25. 25. ELASTIC SCALE OUTHOW TO SCALE UP, OUT OR DOWN IN WINDOWS AZURE - JUAN DE ABREU - #CSwebinar
  26. 26. STATIC CONTENT GENERATION Compute Compute Inactivity Period Average Usage Average Usage Time Time On & off workloads (e.g. batch job) Successful services needs to grow/scale Over provisioned capacity is wasted Keeping up w/ growth is big IT challenge Time to market can be cumbersome Compute Cannot provision hardware fast enough Compute Average Usage Average Usage Time Time Unexpected/unplanned peak in demand Services with micro seasonality trends Sudden spike impacts performance Peaks due to periodic increased demand Can’t over provision for extreme cases IT complexity and wasted capacity HOW TO SCALE UP, OUT OR DOWN IN WINDOWS AZURE - JUAN DE ABREU - #CSwebinar
  27. 27. DEALING WITH VARIABLE LOAD• DEALING WITH VARIABLE LOAD TAKES TWO FORMS• MAINTAINING EXCESS CAPACITY OR HEADROOM Costs: paying for unused capacity Faster availability Asynchronous work pattern can provide buffer• ADDING/REMOVING ADDITIONAL CAPACITY Takes time to spin up Requires management- human or automated Pre-emptive or metric driven HOW TO SCALE UP, OUT OR DOWN IN WINDOWS AZURE - JUAN DE ABREU - #CSwebinar
  28. 28. HEAD ROOM IN WINDOWS AZURE• WEB ROLES Run additional web roles Handle additional load before performance degrades• WORKER ROLES If possible just buffer into queues Will be driven by tolerable level of latency Start additional roles only if queues not clearing Use generic workers to pool resources HOW TO SCALE UP, OUT OR DOWN IN WINDOWS AZURE - JUAN DE ABREU - #CSwebinar
  29. 29. RULE BASED SCALING• USE SERVICE MANAGEMENT AND DIAGNOSTICS APIS• ON/OFF AND PREDICTABLE BURSTING Time based rules• UNPREDICTABLE DEMAND AND FAST GROWTH Monitor metrics and react accordingly HOW TO SCALE UP, OUT OR DOWN IN WINDOWS AZURE - JUAN DE ABREU - #CSwebinar
  30. 30. MONITOR METRICS• PRIMARY METRICS (ACTUAL WORK DONE) Requests per Second Queue messages processed / interval• SECONDARY METRICS CPU Utilization Queue length Response time• DERIVATIVE METRICS Rate of change of queue length Use ‘historical’ data to help predict requirements HOW TO SCALE UP, OUT OR DOWN IN WINDOWS AZURE - JUAN DE ABREU - #CSwebinar
  31. 31. GATHERING METRICS• USE MICROSOFT.WINDOWSAZURE.DIAGNOSTICS.*• CAPTURE VARIOUS METRICS VIA MANAGEMENT API Diagnostics Infrastructure Logs Event Logs Performance Counters IIS Logs• MAY NEED TO SMOOTH/AVERAGE SOME MEASURES• REMEMBER THE COST OF GATHERING DATA Both performance and financial costs Would you use Performance Counters 24/7 on a production system? http://technet.microsoft.com/en-us/library/cc938553.aspx HOW TO SCALE UP, OUT OR DOWN IN WINDOWS AZURE - JUAN DE ABREU - #CSwebinar
  32. 32. EVALUATING BUSINESS RULES• ARE REQUESTS TAKING TOO LONG?• DO I HAVE TOO MANY JOBS IN MY QUEUE?• HOW MUCH MONEY HAVE I SPENT THIS MONTH?• COULD WRITE THESE INTO CODE.• COULD BUILD SOME SORT OF RULES ENGINE.• COULD USE THE WF RULES ENGINE. HOW TO SCALE UP, OUT OR DOWN IN WINDOWS AZURE - JUAN DE ABREU - #CSwebinar
  33. 33. TAKE ACTION• ADD/REMOVE INSTANCES Use Service Management API• CHANGE ROLE SIZE Requires change to *.csdef Most suited to Worker Roles• SEND NOTIFICATIONS Email IM• MANAGE MOMENTUM Be careful not to overshoot HOW TO SCALE UP, OUT OR DOWN IN WINDOWS AZURE - JUAN DE ABREU - #CSwebinar
  34. 34. SUMMARY• DESIGNING FOR MULTIPLE INSTANCES PROVIDES Scale out Availability Elasticity options• CACHING SHOULD BE A KEY COMPONENT OF ANY WINDOWS AZURE APPLICATION• VARIOUS OPTIONS FOR VARIABLE LOAD Spare capacity Scale Out/Back Automation possible HOW TO SCALE UP, OUT OR DOWN IN WINDOWS AZURE - JUAN DE ABREU - #CSwebinar
  35. 35. ANY QUESTIONS?HOW TO SCALE UP, OUT OR DOWN IN WINDOWS AZURE - JUAN DE ABREU - #CSwebinar
  36. 36. THANKS, JUAN DE ABREU VP, Sales Director jdeabreu@getcs.com @juandeabreu http://www.linkedin.com/in/juandeabreuHOW TO SCALE UP, OUT OR DOWN IN WINDOWS AZURE - JUAN DE ABREU - #CSwebinar

×