• Share
  • Email
  • Embed
  • Like
  • Private Content
Webinar issues we_find_slideshare
 

Webinar issues we_find_slideshare

on

  • 335 views

SOASTA's tens-of-thousands of tests in customer labs and production environments have uncovered issues that range from code level bugs to issues in 3rd party services. Testing early, often and at ...

SOASTA's tens-of-thousands of tests in customer labs and production environments have uncovered issues that range from code level bugs to issues in 3rd party services. Testing early, often and at real scale is the only way to be fully prepared.

Statistics

Views

Total Views
335
Views on SlideShare
335
Embed Views
0

Actions

Likes
0
Downloads
8
Comments
0

0 Embeds 0

No embeds

Accessibility

Categories

Upload Details

Uploaded via as Microsoft PowerPoint

Usage Rights

© All Rights Reserved

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Processing…
Post Comment
Edit your comment
  • Good Morning and welcome to the second in our new series, Best Practices in Performance Testing. This series is designed to share with you the best practices our performance engineering team is developing as they work with world-class customers of all sizes. Today we will be exploring what issues are uncovered as customers test applications starting a small scale in development and moving to full-scale testing in production. My name is Betsy Zikakis, I’m the VP, Marketing for Soasta and I’ll be your MC today.
  • Thank you so much for attending this webinar. The archive will be available on our website by the end of this week. Please tell others about it. Our next webinar in this series is focused on testing mobile applications. You can go to the link on this slide to register now. It will be up on the SOASTA home page to register tomorrow morning. And you will receive an email invitation next week.

Webinar issues we_find_slideshare Webinar issues we_find_slideshare Presentation Transcript

  • Best Practices inPerformance Testing Issues Uncovered as Scale IncreasesWebinar Series
  • The Test LabGoals:• Isolate bugs in code• Optimize page performance• Plug memory leaks• Tune databases• Establish performance expectationsLimitations:• Production-quality hardware• IT support and resources• Bandwidth and network infrastructure•3rd party application components• Time
  • The Staging EnvironmentGoals:• Verify capacity• Establish configurations• Stress available resources• Verify performance expectationsLimitations:• Availability / scheduling• Not full-scale• Excludes key components• Time
  • The Full Production EnvironmentGoals:• Test to and beyond expected limits• Verify entire infrastructure• Confirm load balancing and firewall integrity• Certify performance in normal and extreme conditionsBenefits:• All components• 3rd party apps• All integrations• Continuous• Global latencies• Reality
  • Performance isn’t One-Dimensional“You have a fever.” Now what? External web load User Experience QA OPS
  • Performance is Multi-DimensionalFind problems fast when metrics are viewed in context, at scale External and Internal user load Complex Web Applications : AJAX, HTML5, video, mobile, web services User Experience Multi-Dimensional Views
  • Performance is Multi-DimensionalFind problems fast when metrics are viewed in context, at scale External and Internal user load Complex Web Applications : AJAX, HTML5, video, mobile, web services Application Deployment Deployment Component & Infrastructure Infrastructure Network User Experience Code (Internal) (External) Extract Transport Load Real-Time End to End The CloudTest Performance Data platform is built for scalability and real- In-Memory time identification OLAP of issues Multi-Dimensional Views
  • Extending Performance Beyond the Lab Discovering issues across teams, time and scaleStage / Team Debugging code Garbage collection Memory leaks Memory leaks Database thread counts Dev & Test Slow pages Inefficient database queries Method-level tuning Test Lab (a few thousand users) Scale of Test
  • Extending Performance Beyond the Lab Discovering issues across teams, time and scaleStage / Team Configuration Setting & Sizing Auto-scaling failures Release Max sockets exceeded Global latency variance & Deploy Slow third-party plug-ins Security bottlenecks Default configuration settings Search technology limits Inadequate server resources Garbage collection Memory leaks Memory leaks Database thread counts Dev & Test Slow pages Inefficient database queries Method-level tuning Test Lab Staging (thousands to 10’s of thousands) Scale of Test
  • Extending Performance Beyond the Lab Discovering issues across teams, time and scale Tuning for full-scale…and well beyond Latency between systems Network Network configuration & Ops Network bandwidth CDN file placement Conflict with other apps DNS routingStage / Team Load Balancer configuration Firewall max capacity Auto-scaling failures Unbalanced web servers Release & Deploy Max sockets exceeded s Global latency variance Security bottlenecks Slow third-party plug-ins Default configuration settings Search technology limits Inadequate server resources Garbage collection Memory leaks Memory leaks Database thread counts Dev & Test Slow pages Inefficient database queries Method-level tuning Test Lab Staging Production (100% +++) Scale of Test
  • CloudTest LiveReal-world Issue ResolutionCompany Profile Project ProfileCompany Toy Manufacturer Environment Lab and StagingSize Fortune 500 Application Drupal (Apache, PHP, MySQL) Location Private Hosted Cloud Objective Identify Breaking Points Project Length Five Days (non-consecutive) Lab Staging
  • CloudTest Live Real-world Issue Resolution• First test cycle in the lab • Some servers hitting 100% CPU Lab• 239 hits per second • Bottleneck in throughput• 40.5 MBit/second bandwidth
  • CloudTest Live Real-world Issue Resolution• Second test cycle in the lab • Some servers hitting 100% CPU Lab• Higher concurrency target • Performance problem over time• Bandwidth stayed the same • Load balancer flapping servers
  • CloudTest Live Real-world Issue Resolution• Duration problem was crippling – could not recover Lab
  • CloudTest Live Real-world Issue Resolution• Bandwidth – ouch! Staging• CPU at 100%
  • CloudTest Live Real-world Issue Resolution• 767 Home pages served at 650 megs total transfer.• That’s an 850kb page weight! Yikes! Staging
  • CloudTest LiveReal-world Issue Resolution Mission Debriefing - Fortune 500 Toy Manufacturer Lab Staging• Poor throughput on the initial tests • Massive page weight on the home page• CPU hits 100% • Application unresponsive at ~2000 users• Bandwidth cap identified in the lab• Database connections leaking over time• Load balancer dropping servers Outcome • Customer happy with the breaking point identified • Optimized web assets in the home page • Offloaded assets to the CDN • Load balancer algorithm optimizations
  • CloudTest LiveReal-world Issue ResolutionCompany Profile Project ProfileCompany Media and Broadcasting Environment ProductionSize Fortune 500 Application Java Location Private Data Center Objective 25,000 Concurrent Project Length Three Days Production
  • CloudTest Live Real-world Issue Resolution• Steadily increasing response times• Constant stream of errors (< 1%) Production• Variety of error types
  • CloudTest LiveReal-world Issue Resolution• One rogue page – streaming the TV episode Production
  • CloudTest LiveReal-world Issue Resolution• Connection reset errors starting the video stream• Entitlement license server calls failing Production
  • CloudTest LiveReal-world Issue Resolution Mission Debriefing - Fortune 500 Media Company Production Outcome• Connection issues with CDN • Worked with CDN provider to reinforce certain geographies• One problem page • Added capacity to the license server cluster• Massive bandwidth usage
  • CloudTest LiveReal-world Issue ResolutionCompany Profile Project Profile Company Advanced Technology Environment Production Size Fortune 50 Application Java (Struts), Oracle Multiple HostedPrivate Location Data Centers Objective 172,000 Concurrent Project Length Two Months Production
  • CloudTest LiveReal-world Issue Resolution • Recurring error pattern during scenario completion Production
  • CloudTest LiveReal-world Issue Resolution Awesome outcome in Production!
  • CloudTest LiveReal-world Issue Resolution Mission Debriefing - Fortune 50 Technology Company Production Outcome• Load balancer firmware bug with 2048 bit cert • Worked with hardware LB manufacturer on a fix• Database issues with the login process • Really low error rate • Successful launch!
  • Q&APlease join us for the next webinar in this series: Testing Mobile Applications May 3rd at 10AM PDT, 1PM EDT Register at: http://bit.ly/ezq3fP Contact SOASTA: www.soasta.com/cloudtest/ info@soasta.com 866.344.8766 Follow us: twitter.com/cloudtest facebook.com/cloudtest