Windows AzureAcid Test!DDD Sydney,  2 July 2011<br />Matt Done, expanz Pty. Ltd.<br />
Agenda<br />Presentation Overview<br />What is Azure and SQL Azure<br />Goals of the Load Test<br />Testing Setup<br />Tes...
Presentation Overview<br />Topics<br />Load testing in Windows Azure and the various benefits it brings<br />Horizontally ...
What is Windows Azure<br /><ul><li>Microsoft Cloud Based Platform as a Service (PaaS)</li></li></ul><li>What is SQL Azure<...
Have an environment to tweak performance settings and to find potential bottlenecks.
To assess how the Azure platform performs under load.
To assess how SQL azure performs under load.</li></li></ul><li>Application Architecture<br />
Load Testing Architecture<br />
Azure Load Test Costs<br />Load Testing Setup Cost<br />Total Cost = $15.71 per hour<br />
Local Debug<br />
Local/Remote Debug<br />Problem<br /><ul><li>Need to test and locally debug
Azure will not accept remote connections</li></ul>Solution<br />Install SSH server (Putty, BitVise)<br />Start azure compu...
(Very) Simple Banking App<br />
Load Test Setup<br />
Database Setup<br />Table Sizing<br />Indexes<br /><ul><li>Non Clustered Index on [dbo].[Transactions], Columns account_id...
Non Clustered Index on [dbo].[Transactions], Columns [billingaddressbook_id]
Non Clustered Index on [dbo].[Transactions], Columns [account_id]
Non Clustered Index on [dbo].[BillingAddressBooks], Columns [user_id]</li></ul>Considerations<br /><ul><li>Make sure the D...
Allocate percentage execution for each process
Determine wait time figure (100ms or 1 sec)?
Each execution waits a random amount of time, based on processExecutionDelay. Thread.Sleep(myRandom.Next(processExecutionD...
Each User session is a single thread. 2500 users = 2500 threads.</li></li></ul><li>Web Service Request/Response Capture<br...
Do not log to file system
Preserve thread pool threads for load test execution
Preserve bandwidth and CPU as little as possible</li></ul>Solution<br /><ul><li>Use SQL Azure table storage
Capture Request/Responses and send to Site Manager
Upcoming SlideShare
Loading in...5
×

Windows Azure Acid Test

1,723

Published on

Presentation delivered by Matt Done, Head Of Platform Development at expanz Pty. Ltd. during DDD Sydney event on 2 July 2011.
Matt demonstrates what it takes to setup a highly sophisticated load test, using the Azure environment and how to use the results to optimise a fully blown application development platform and application server running on Azure.
Recording of this presentation can be found at www.youtube.com/expanzTV

Published in: Technology
1 Comment
0 Likes
Statistics
Notes
  • recorded video can be found here: http://vimeo.com/26261446
       Reply 
    Are you sure you want to  Yes  No
    Your message goes here
  • Be the first to like this

No Downloads
Views
Total Views
1,723
On Slideshare
0
From Embeds
0
Number of Embeds
2
Actions
Shares
0
Downloads
13
Comments
1
Likes
0
Embeds 0
No embeds

No notes for slide
  • DB Transactions are free for internal apps. Data transfer is 10c per 1gb. MSDN includes 5gb
  • vvvv
  • During our Private Beta Program we are looking for thought leading developers for trialling an exclusive preview of our next generation Line Of Business (LOB) Application Development Platform, the ‘expanzPLATFORM 2011’ (eP).Though we’re a small Aussie start-up, we’ve achieved some pretty cool stuff already. Analyst RecognitionOne of six companies global by Gartner as a ‘cool vendor’.Proof Of Conceptwe built and run a multi-tenant, multi device, multi general ledger ERP SaaS app with eP which is live on Azure and used by large Australian distribution companies. Mission Critical ProjectLogica built and deployed the app that counted all votes for the recent NSW 2011 State Election, using eP.Our core values to developers areProductivity extreme code reuse / inheritancemetadata driven framework (rather than code gen)business logic is all on the server side and supports any clientPortability Client – Native Thin UX on any client/device Server - Premise/Hosted/Cloud (Windows)If you believe great design and architecture matters, that writing code (preferably once) is the best way to handle complexity, then you’ll enjoy getting under our hood. To join and qualify for your free perpetual license, please email us at beta@expanz.com with ‘DDD Attendee’ in your subject line before 22 July 2011.
  • Windows Azure Acid Test

    1. 1. Windows AzureAcid Test!DDD Sydney, 2 July 2011<br />Matt Done, expanz Pty. Ltd.<br />
    2. 2. Agenda<br />Presentation Overview<br />What is Azure and SQL Azure<br />Goals of the Load Test<br />Testing Setup<br />Test Results<br />Cost Comparisons<br />Live Demo<br />Questions<br />
    3. 3. Presentation Overview<br />Topics<br />Load testing in Windows Azure and the various benefits it brings<br />Horizontally Scalable, Stateful computing.<br />High performance WCF (Windows Communications Foundation)<br />Intended Audience<br />Enterprise Architects, Solution Architects<br />Developers involved in large scale applications<br />People who love Azure!!<br />
    4. 4. What is Windows Azure<br /><ul><li>Microsoft Cloud Based Platform as a Service (PaaS)</li></li></ul><li>What is SQL Azure<br /><ul><li>Microsoft Cloud Based Database as a Service (DBaaS)</li></li></ul><li>Goals of the load test<br /><ul><li>To prove the efficiency of the expanz Platform under load of 10000+ concurrent users.
    5. 5. Have an environment to tweak performance settings and to find potential bottlenecks.
    6. 6. To assess how the Azure platform performs under load.
    7. 7. To assess how SQL azure performs under load.</li></li></ul><li>Application Architecture<br />
    8. 8. Load Testing Architecture<br />
    9. 9. Azure Load Test Costs<br />Load Testing Setup Cost<br />Total Cost = $15.71 per hour<br />
    10. 10. Local Debug<br />
    11. 11. Local/Remote Debug<br />Problem<br /><ul><li>Need to test and locally debug
    12. 12. Azure will not accept remote connections</li></ul>Solution<br />Install SSH server (Putty, BitVise)<br />Start azure compute emulator<br />Start local client<br />Start SSH client and port forward to Web service port on remote machine<br />Start testing<br />
    13. 13. (Very) Simple Banking App<br />
    14. 14. Load Test Setup<br />
    15. 15. Database Setup<br />Table Sizing<br />Indexes<br /><ul><li>Non Clustered Index on [dbo].[Transactions], Columns account_id, [transactiondate]
    16. 16. Non Clustered Index on [dbo].[Transactions], Columns [billingaddressbook_id]
    17. 17. Non Clustered Index on [dbo].[Transactions], Columns [account_id]
    18. 18. Non Clustered Index on [dbo].[BillingAddressBooks], Columns [user_id]</li></ul>Considerations<br /><ul><li>Make sure the Database resides at the same location as your application</li></li></ul><li>Load Test Execution<br /><ul><li>Record trace of required processes
    19. 19. Allocate percentage execution for each process
    20. 20. Determine wait time figure (100ms or 1 sec)?
    21. 21. Each execution waits a random amount of time, based on processExecutionDelay. Thread.Sleep(myRandom.Next(processExecutionDelay));
    22. 22. Each User session is a single thread. 2500 users = 2500 threads.</li></li></ul><li>Web Service Request/Response Capture<br />DesignObjectives<br /><ul><li>Do not log to SQL Azure database
    23. 23. Do not log to file system
    24. 24. Preserve thread pool threads for load test execution
    25. 25. Preserve bandwidth and CPU as little as possible</li></ul>Solution<br /><ul><li>Use SQL Azure table storage
    26. 26. Capture Request/Responses and send to Site Manager
    27. 27. Site Manager will process and commit records
    28. 28. Use actual threads and not thread pool threads</li></ul>Issues<br /><ul><li>Batch table storage requests in lots of 20-50 and use SaveChangesWithRetries
    29. 29. Use Thread.Startinstead of  ThreadPool.QueueUserWorkItem
    30. 30. Datetime.Now only accurate to 15-20ms. Use StopWatchclass</li></li></ul><li>Capturing Diagnostics in Azure<br /><ul><li>Azure has an API for logging diagnostic information to TableStorage
    31. 31. Counter names are normal windows counter. Eg@"Processor(_Total)% Processor Time" , @"MemoryAvailable MBytes"
    32. 32. Data can be viewed with Cerebrata Diagnostics Studio (Demo)</li></li></ul><li>Load Test Results (First Cut)<br />Application Instances Setup<br />Load Test Instance Setup<br /><ul><li>1 Extra Large Instance
    33. 33. 4 Geographic locations
    34. 34. 5 Large Web Services
    35. 35. 1 Large Site Manager
    36. 36. 5 Large Session Managers</li></ul>Large Instance = 4 x 1.6 GHz cpu’sand 7 GB of ram<br />Test Result Summary<br />Run duration = 32 Minutes<br />Total Transactions = 1.89 Million<br />Average Number of Transactions per Minute = 59767<br />Average WS Response Time = 70ms<br />Average CPU Utilisation (App Servers) = 30%<br />Min Memory (App Servers) = 4Gb<br />NOT GOOD ENOUGH. NEED MORE!!!<br />
    37. 37. Load Test First Cut Investigation<br /><ul><li>WCF Web services are not multithreaded by default</li></ul>[ServiceBehavior(ConcurrencyMode = ConcurrencyMode.Multiple)] <br /><ul><li>When enabling multiple concurrency, throttling is important</li></ul> <serviceThrottling maxConcurrentSessions="400"  maxConcurrentInstances="400" maxConcurrentCalls="400" /><br /><ul><li>When using Channel Factories, max number of open concurrent connections is 10.
    38. 38. Adjust max concurrent connections when using TCP or Named pipe bindings, is 2.</li></ul><system.net>   <br />  <connectionManagement>      <br /> <add address="*" maxconnection="500"/>   <br />  </connectionManagement> <br /></system.net><br /><ul><li>Azure load balancer is not throttling connections. Throttle Connections using ServicePointManager.DefaultConnectionLimit in Web Role
    39. 39. Avoid closing factory connections in a finally block. Or do both.</li></li></ul><li>The channel factory, factory<br />Problem<br /><ul><li>Limit of 10 open connections.
    40. 40. For singleton to singleton operations, comms is problematic
    41. 41. Aborted and faulted channels</li></ul>Solution<br /><ul><li>Create a factory of channel factories(pooling)
    42. 42. Needs to be Muti-threaded.</li></ul>Couple of Cool and not so cool things (Code)<br /><ul><li>Parallel Linq (PLinq)
    43. 43. To get a Service Channel Guid Use the following </li></ul> ((IClientChannel)channel).SessionId<br /><ul><li>Concurrent Collections. Not so cool.</li></li></ul><li>Run 2,3,4…<br />Application Instances Setup<br />Load Test Instance Setup<br /><ul><li>5 Medium Instances
    44. 44. 4 Geographic locations
    45. 45. 10 Large Web Services
    46. 46. 1 Extra Large Site Manager
    47. 47. 8 Large Session Managers</li></ul>Large Instance = 4x 1.6 GHz cpu’sand 7 GB of ram<br />Test Result Summary<br />Run duration = 30 Minutes<br />Total Transactions = 7.8 Million<br />Average Number of Transactions per Minute = 161245<br />Max Transactions per Minute = 211589<br />Error Rate = 0.014%<br />Average Web Service response time = 39ms<br />Average CPU Utilisation (App Servers) = 70%<br />Min Memory (App Servers) = 4Gb<br />Total Cost = $12.88 per hour<br />
    48. 48. Headline Results<br />
    49. 49. Time line<br />Default WCF Settings. 5 Instances<br />Default WCF Settings. 10 Instances<br />Multithreaded Service. Per Session Context<br />Create Channel Factory Factory<br />Raise tcp Connections limit<br />Adjust Instance Counts<br />
    50. 50. Demo<br /><ul><li>Live Diagnostics
    51. 51. Live Table Storage
    52. 52. Live working client
    53. 53. Please find full recording of my session at</li></ul>www.youtube.com/expanzTV<br />
    54. 54. Thank you for following my presentation!<br />Please take the time to have a look at the next slide with an exclusive invitation for DDD attendees from expanz as a DDD sponsor.<br />
    55. 55. BETA<br />Perpetual License exclusively for DDD Attendees<br />During our Private Beta Program expanz are looking for thought leading developers for trialling an exclusive preview of their next generation Line Of Business (LOB) Application Development Platform, the ‘expanzPLATFORM 2011’ <br />As a DDD Melbourne, Sydney or Adelaide attendee, you qualify for participating in our Private Beta program. To join and receive your free perpetual license after launch, please email us at beta@expanz.com with <br />‘DDD Attendee’ in your subject line before 22 July 2011. <br />
    1. A particular slide catching your eye?

      Clipping is a handy way to collect important slides you want to go back to later.

    ×