Production Performance Testing in the Cloud
Upcoming SlideShare
Loading in...5

Production Performance Testing in the Cloud



Testing in production for online applications has evolved into a critical component of successful performance testing strategies. Dan Bartow explains the fundamentals of cloud computing, its ...

Testing in production for online applications has evolved into a critical component of successful performance testing strategies. Dan Bartow explains the fundamentals of cloud computing, its application to full-scale performance validation, and the practices and techniques needed to design and execute a successful testing-in-production strategy. Drawing on his experiences, Dan describes the methodology he has used for testing numerous online applications in a production environment with minimal disruption. He explains how to create a performance testing strategy to give your team critical data about how your online application performs and scales. Learn how to create a robust lab-to-production ecosystem that delivers the answers about what will happen when peak traffic hits your site. Take back practical approaches to mitigate the three most common problems—security, test data, and potential live customer impact—that arise when embarking on testing in production.



Total Views
Views on SlideShare
Embed Views



2 Embeds 8 7 1



Upload Details

Uploaded via as Adobe PDF

Usage Rights

© All Rights Reserved

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
Post Comment
Edit your comment

Production Performance Testing in the Cloud Document Transcript

  • 1. TK PM Tutorial 10/1/2013 1:00:00 PM "Production Performance Testing in the Cloud" Presented by: Dan Bartow SOASTA Brought to you by: 340 Corporate Way, Suite 300, Orange Park, FL 32073 888-268-8770 ∙ 904-278-0524 ∙ ∙
  • 2. Dan Bartow SOASTA, Inc. At SOASTA, Dan Bartow is Vice President of Product Management for the industry leading and award-winning product CloudTest. Prior to joining SOASTA, Dan was Senior Manager of Engineering at Intuit where his team was responsible for the speed and stability of TurboTax Online. During the past decade he has been responsible for the performance of websites for dozens of leading consumer brands.
  • 3. 8/27/2013 Create a One-Page Capacity Model for High-Traffic Web Applications Dan Bartow SOASTA About the Speaker SOASTA VP Product Management CloudTest Evangelist Intuit Sr. Manger, Engineering TurboTax Online and E-com ATG Sr. Deployment Engineer Work American Airlines, Best Buy, Target, Turbotax Online, QuickenOnline, MySpace, Dennys, Dominos, Mattel, Hallmark, FAA, US Army, AT&T Wireless, Alcatel, Newsweek, Oprah, NeimanMarcus, SBC, Plantronics, Kodak, JCrew, Cingular, Newell Rubbermaid 1
  • 4. 8/27/2013 Poor performance. The Problems 1. No knowledge of what the critical metrics are at each tier of an applications architecture 1. No record of what the critical metric values were at the peak last year 1. No idea of what those metrics should be at the next peak traffic day 1. No indication of what the capacity is today and how that relates to the upcoming peak 2
  • 5. 8/27/2013 The Solution (or at least a huge step in the right direction) The one page capacity model What one looks like 3
  • 6. 8/27/2013 How to build it Step 1. Create a simple architecture diagram, containing mostly infrastructure, that lists key technologies in play for each tier You don’t need this 4
  • 7. 8/27/2013 Getting closer What to include in the diagram 1. 2. 3. Key infrastructure components Critical services and their infrastructure where possible Third party components (if applicable) Hint: If you can’t test it, you probably don’t need to include it. #controversial 5
  • 8. 8/27/2013 A reasonable diagram How to build it Step 2. Identify the metrics that should be tracked for each tier 6
  • 9. 8/27/2013 Three layers of monitoring Layer 1 - Customer Concurrent sessions Pages (requests) / sec Layer 2 - Server CPU % Memory Network IO Disk IO Layer 3 - Container JVM Heap usage Threads / second Login servlet invocations / sec Database connections To make this all happen (and be really useful) You need comprehensive monitoring You need to be able able to test to scale 7
  • 10. 8/27/2013 Metrics examples for common technologies Metric examples for common technologies CPU Memory TX/sec (especially SSL TX) Bandwidth/sec CPU Memory Threads / sec Container Application CPU Memory Threads or processes / sec Key GET/POST actions Key method calls External connections Container Application CPU Memory JVM heap usage Garbage collection interval Threads / sec Key servlet invocation count (login, order placement) Database connection count Key servlet execution time Container CPU Memory Connections Queries / sec (SELECT/INSERT/UPDATE/ DELETE) Table or row locks 8
  • 11. 8/27/2013 How to build it Step 3. Fill in the values for your peak last year (the peak second) How to build it Step 4. Project and do simple math for the upcoming year. Base this math on top level percentages and honor relationships between metrics where needed. 9
  • 12. 8/27/2013 Contact Information Dan Bartow VP Product Management SOASTA Inc. Email: Twitter: LinkedIn: Blog: 10