Your SlideShare is downloading. ×
  • Like
The Cloud: A game changer to test, at scale and in production,  SOA based web and mobile applications.
Upcoming SlideShare
Loading in...5

Thanks for flagging this SlideShare!

Oops! An error has occurred.


Now you can save presentations on your phone or tablet

Available for both IPhone and Android

Text the download link to your phone

Standard text messaging rates apply

The Cloud: A game changer to test, at scale and in production, SOA based web and mobile applications.


Today in retail, financial services, media, telecommunications and a host of other industries, more and more business is transacted through consumer web sites and mobile applications. With new …

Today in retail, financial services, media, telecommunications and a host of other industries, more and more business is transacted through consumer web sites and mobile applications. With new channels creating spikes in traffic, highly complex system architectures, and internet-savvy customers, websites and web applications must be tested at scale to maximize business results and avoid a catastrophic crash. However, whether due to time or cost or other reasons, upwards of 90 percent of web applications are not fully tested before launching. If testing is done, many times it's with a small percent of expected traffic, which is then extrapolated for an estimation of performance.

Cloud computing is changing the game for testing web applications. Cloud testing enables, for the first time, performance testing that complements the lab and accounts for the conditions in a production environment, such as traffic spikes, network latency, firewalls, and other factors. And it can be done far more affordably than traditional testing methods, as part of agile development cycles, and without an army of highly skilled performance engineers.

Presentation given in Rome for the International SOA conference - Moving SOA into the Cloud in Rome, May 2011

Published in Technology
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Be the first to comment
No Downloads


Total Views
On SlideShare
From Embeds
Number of Embeds



Embeds 0

No embeds

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

    No notes for slide


  • 1. The CloudA game changer to test, at scale and in production, SOA based web and mobile applications. Fred Beringer – VP Business Development Europe – SOASTA International SOA Conference 2011- Rome
  • 2. PERFORMANCEmatters more than ever before
  • 3. #1 highly viralSocial media isand drives massive amount oftraffic.
  • 4. 500 million active users spending 700 billion minutes per month
  • 5. 13 million hours uploaded in 2010 2 billion video viewed per day
  • 6. 106 million accounts10 billion tweets in 2010
  • 7. Your new marketing mix
  • 8. #2Web Architecture is complex
  • 9. Inside and outside the firewall Analytics Switch Ad Server Databases Search Session Info CDN Content CMS ContentShopping Cart Media Server Application servers Load balancers Payment Server
  • 10. #3Content is king
  • 11. 1980
  • 12. 1992
  • 13. 1996
  • 14. 2011
  • 15. #4Access devices explosion
  • 16. Different performance on each device
  • 18. 39% of consumers say speed is more important than functionality Source: Forrester Research
  • 19. 47% Expect a web page to load in two seconds or less Source: Forrester Research
  • 20. Will abandon a web page if it40% takes more than three seconds to load Source: Forrester Research
  • 21. #6Low performance and unreliability bringsMASSIVE BUSINESS IMPACT
  • 22. PayPal down 45 minutes in 2009 $45 Millions lost revenue
  • 23. Amazon loses 1% of their sales for each 100ms of latency
  • 24. Google loses 20% of search trafficevery 500ms of additional latency
  • 25. 60% of companies admit that they do not have any performance management processes in place or what they have is ineffective. DynaTrace study- 2009
  • 26. Of companies having performance management50% processes admit that they are doing it only in reactive way when problem occurs. DynaTrace study- 2009
  • 27. 50% of application problems are found in production – when those problems have impacted end users and are most expensive to resolve. DynaTrace study- 2009
  • 28. What is required today to test effectively performance of web applications?
  • 29. #1SCALE
  • 30. #2Real time analysis of performance data
  • 31. #3SPEED
  • 33. Ready for some pain?
  • 34. Traditional Performance Testing doesn’t work anymore
  • 35. 2006
  • 37. Cloud TestingThe massive resources of the cloud have changed thegame for performance testing. Cloud Testing enablesorganizations to test their consumer facing Web andmobile applications fast, affordably and at scale.There’s now no excuse for the type of Web site disasterswe’ve seen in the past that cost businesses millions inrevenues and perhaps more in reputation. Tom Lounibos CEO SOASTA
  • 38. What does it bring us?
  • 39. #1Affordable platform for infinite load
  • 40. Billions possible points of failureThe new testing challenge
  • 41. #2Real-time analysis brings wisdom
  • 42. #3Global and realistic traffic
  • 43. #4Where is the needle?
  • 44. In your code
  • 45. In your infrastructure
  • 46. Outside your firewall
  • 47. Extending Performance Beyond the Lab Discovering Issues across teams, time and scale Latency between systems Network Network configuration & Ops Network bandwidth CDN file placement Conflict with other apps DNS routing Load Balancer configurationStage / Team Firewall max capacity Auto-scaling failures Unbalanced web servers Release & Deploy Max sockets exceeded s Global latency variance Security bottlenecks Slow third-party plug-ins Default configuration settings Search technology limits Inadequate server resources Garbage collection Memory leaks Memory leaks Database thread counts Dev & Test Slow pages Inefficient database queries Method-level tuning Test Lab (<10k Vus) Staging (10-100k) Production (> 100k) Scale of Test
  • 48. CloudTest Live Real-world Issue ResolutionCompany Profile Project ProfileCompany Toy Manufacturer Environment Lab and StagingSize Fortune 500 Application Drupal (Apache, PHP, MySQL) Location Private Hosted Cloud Objective Identify Breaking Points Project Length Five Days (non-consecutive) Lab Staging
  • 49. CloudTest Live Lab Real-world Issue Resolution•  First test cycle in the lab •  Some servers hitting 100% CPU•  239 hits per second •  Bottleneck in throughput•  40.5 MBit/second bandwidth
  • 50. CloudTest Live Lab Real-world Issue Resolution•  Second test cycle in the lab •  Some servers hitting 100% CPU•  Higher concurrency target •  Performance problem over time•  Bandwidth stayed the same •  Load balancer flapping servers
  • 51. CloudTest Live Staging Real-world Issue Resolution•  Bandwidth (238Mbits/s) – ouch!•  CPU at 100%
  • 52. CloudTest Live Staging Real-world Issue Resolution•  767 Home pages served at 650 megs total transfer.•  That’s an 850kb page weight! Yikes!
  • 53. CloudTest Live Real-world Issue Resolution •  Poor throughput on the initial tests Lab •  CPU hits 100% •  Bandwidth cap identified in the lab •  Database connections leaking over time •  Load balancer dropping servers Staging •  Massive page weight on the home page •  Application unresponsive at ~2000 users•  Customer happy with the breaking point identified•  Optimized web assets in the home page•  Offloaded assets to the CDN•  Load balancer algorithm optimizations
  • 54. CloudTest Live Real-world Issue ResolutionCompany Profile Project ProfileCompany Advanced Technology Environment ProductionSize Fortune 50 Application Java (Struts), Oracle Multiple Hosted Private Location Data Centers Objective 172,000 Concurrent Project Length Two Months Production
  • 55. CloudTest Live Production Real-world Issue ResolutionRecurring error pattern during scenario completion
  • 56. CloudTest Live Real-world Issue ResolutionAwesome outcome in Production!
  • 57. CloudTest Live Real-world Issue ResolutionProduction •  Load balancer firmware bug with 2048 bit cert •  Database issues with the login process •  Worked with hardware LB manufacturer on a fix •  Really low error rate •  Successful launch!
  • 58. Fit for an #5 AGILE WORLD
  • 59. Real-time collaboration
  • 60. Continuous performance testing
  • 61. Cloud Testing helps companies
  • 62. •  Every day at noon is a holiday spike in traffic.•  Pages must render to the user in under 1 second.•  50% of all revenue from Gilt sales comes in less than 1 hour after sale starts.
  • 63. Cloud Testing helps dolls
  • 64. Cloud Testing helps taxes
  • 65. Cloud Testing helps phones
  • 66. Testing SOA in the cloudCHALLENGES
  • 67. It gets crowded
  • 68. Loose SLAs
  • 69. Harder to predict
  • 70. (slightly)Different strategy
  • 71. Same challenges • Services with no user interface • Data driven • Services outside the organization/cloud • End-to-end scenario • Test ownershipSame Bottom-up performance testing approach • Individual services • Set (integration) of services • End-to-endDeal with Cloud unpredictability • Varying performance encourage continuous performance testing/monitoring. • Infrastructure upgrade could impact performance. • Expect to baseline on premise application and run benchmark in the cloud after migration. • Test the full bottom-up cycle for increased confidence.
  • 72. Let’s see how this website performs!
  • 73. Questions/Discussion
  • 74. The CloudA game changer for web performance testing
  • 75. Photos Credits:Slide 2: 18, 19, 20: 21: 22,23,24: 25: 32: 34: 46: 48: 54: 59: 60: 62: