0
The CloudA game changer to test, at scale and in production,     SOA based web and mobile applications.         Fred Berin...
PERFORMANCEmatters more than ever before
#1              highly viralSocial media isand drives massive amount oftraffic.
500 million active users  spending 700 billion minutes per month
13 million hours uploaded in 2010   2 billion video viewed per day
106 million accounts10 billion tweets in 2010
Your new marketing mix
#2Web Architecture is complex
Inside and outside the firewall                                      Analytics                     Switch                 ...
#3Content is king
1980
1992
1996
2011
#4Access devices explosion
Different performance on       each device
#5HIGHER EXPECTACTIONS
39%   of consumers say speed is more        important than functionality                              Source: Forrester Re...
47%   Expect a web page to load in          two seconds or less                             Source: Forrester Research
Will abandon a web page if it40%   takes more than three seconds                 to load                              Sour...
#6Low performance and unreliability bringsMASSIVE BUSINESS IMPACT
PayPal down 45 minutes in 2009   $45 Millions lost revenue
Amazon loses 1% of their sales for    each 100ms of latency
Google loses 20% of search trafficevery 500ms of additional latency
60%      of companies admit that they do not have      any performance management processes      in place or what they hav...
Of companies having      performance      management50%   processes admit that      they are doing it only      in reactiv...
50%      of application problems are found in production –      when those problems have impacted end users and      are m...
What is required today to    test effectively  performance of web     applications?
#1SCALE
#2Real time analysis         of performance data
#3SPEED
AFFORDABILITY   #4
Ready for some pain?
Traditional Performance Testing     doesn’t work anymore
2006
PERFORMANCE TESTINGREVOLUTION
Cloud TestingThe massive resources of the cloud have changed thegame for performance testing. Cloud Testing enablesorganiz...
What does it bring us?
#1Affordable platform for infinite load
Billions possible points of failureThe new testing challenge
#2Real-time analysis brings wisdom
#3Global and realistic traffic
#4Where is the needle?
In your code
In your infrastructure
Outside your firewall
Extending Performance Beyond the Lab                                           Discovering Issues across teams, time and s...
CloudTest Live                          Real-world Issue ResolutionCompany Profile                        Project ProfileC...
CloudTest Live                                 Lab                       Real-world Issue Resolution•  First       test cy...
CloudTest Live                                                             Lab                      Real-world Issue Resol...
CloudTest Live                                                   Staging                     Real-world Issue Resolution• ...
CloudTest Live                                                       Staging                    Real-world Issue Resolutio...
CloudTest Live              Real-world Issue Resolution           •  Poor throughput on the initial tests  Lab      •  CPU...
CloudTest Live                         Real-world Issue ResolutionCompany Profile                           Project Profil...
CloudTest Live                                            Production           Real-world Issue ResolutionRecurring error ...
CloudTest Live     Real-world Issue ResolutionAwesome outcome in Production!
CloudTest Live                   Real-world Issue ResolutionProduction             •  Load balancer firmware bug with 2048...
Fit for an   #5  AGILE WORLD
Real-time collaboration
Continuous performance testing
Cloud Testing helps companies
•    Every day at noon is a holiday spike in traffic.•    Pages must render to the user in under 1 second.•    50% of all ...
Cloud Testing helps dolls
Cloud Testing helps taxes
Cloud Testing helps phones
Testing SOA in the cloudCHALLENGES
It gets crowded
Loose SLAs
Harder to predict
(slightly)Different strategy
Same challenges • Services with no user interface • Data driven • Services outside the organization/cloud • End-to-end sce...
Let’s see how this website performs!
Questions/Discussion
The CloudA game changer for web performance testing                 twitter.com/fredberinger                 fberinger@soa...
Photos Credits:Slide 2: http://www.sxc.hu/photo/1219246Slide 18, 19, 20: http://www.sxc.hu/photo/1267744Slide 21: http://w...
The Cloud: A game changer to test, at scale and in production,  SOA based web and mobile applications.
The Cloud: A game changer to test, at scale and in production,  SOA based web and mobile applications.
The Cloud: A game changer to test, at scale and in production,  SOA based web and mobile applications.
The Cloud: A game changer to test, at scale and in production,  SOA based web and mobile applications.
Upcoming SlideShare
Loading in...5
×

The Cloud: A game changer to test, at scale and in production, SOA based web and mobile applications.

1,328

Published on

Today in retail, financial services, media, telecommunications and a host of other industries, more and more business is transacted through consumer web sites and mobile applications. With new channels creating spikes in traffic, highly complex system architectures, and internet-savvy customers, websites and web applications must be tested at scale to maximize business results and avoid a catastrophic crash. However, whether due to time or cost or other reasons, upwards of 90 percent of web applications are not fully tested before launching. If testing is done, many times it's with a small percent of expected traffic, which is then extrapolated for an estimation of performance.

Cloud computing is changing the game for testing web applications. Cloud testing enables, for the first time, performance testing that complements the lab and accounts for the conditions in a production environment, such as traffic spikes, network latency, firewalls, and other factors. And it can be done far more affordably than traditional testing methods, as part of agile development cycles, and without an army of highly skilled performance engineers.

Presentation given in Rome for the International SOA conference - Moving SOA into the Cloud in Rome, May 2011

Published in: Technology
0 Comments
1 Like
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total Views
1,328
On Slideshare
0
From Embeds
0
Number of Embeds
3
Actions
Shares
0
Downloads
33
Comments
0
Likes
1
Embeds 0
No embeds

No notes for slide

Transcript of "The Cloud: A game changer to test, at scale and in production, SOA based web and mobile applications."

  1. 1. The CloudA game changer to test, at scale and in production, SOA based web and mobile applications. Fred Beringer – VP Business Development Europe – SOASTA International SOA Conference 2011- Rome
  2. 2. PERFORMANCEmatters more than ever before
  3. 3. #1 highly viralSocial media isand drives massive amount oftraffic.
  4. 4. 500 million active users spending 700 billion minutes per month
  5. 5. 13 million hours uploaded in 2010 2 billion video viewed per day
  6. 6. 106 million accounts10 billion tweets in 2010
  7. 7. Your new marketing mix
  8. 8. #2Web Architecture is complex
  9. 9. Inside and outside the firewall Analytics Switch Ad Server Databases Search Session Info CDN Content CMS ContentShopping Cart Media Server Application servers Load balancers Payment Server
  10. 10. #3Content is king
  11. 11. 1980
  12. 12. 1992
  13. 13. 1996
  14. 14. 2011
  15. 15. #4Access devices explosion
  16. 16. Different performance on each device
  17. 17. #5HIGHER EXPECTACTIONS
  18. 18. 39% of consumers say speed is more important than functionality Source: Forrester Research
  19. 19. 47% Expect a web page to load in two seconds or less Source: Forrester Research
  20. 20. Will abandon a web page if it40% takes more than three seconds to load Source: Forrester Research
  21. 21. #6Low performance and unreliability bringsMASSIVE BUSINESS IMPACT
  22. 22. PayPal down 45 minutes in 2009 $45 Millions lost revenue
  23. 23. Amazon loses 1% of their sales for each 100ms of latency
  24. 24. Google loses 20% of search trafficevery 500ms of additional latency
  25. 25. 60% of companies admit that they do not have any performance management processes in place or what they have is ineffective. DynaTrace study- 2009
  26. 26. Of companies having performance management50% processes admit that they are doing it only in reactive way when problem occurs. DynaTrace study- 2009
  27. 27. 50% of application problems are found in production – when those problems have impacted end users and are most expensive to resolve. DynaTrace study- 2009
  28. 28. What is required today to test effectively performance of web applications?
  29. 29. #1SCALE
  30. 30. #2Real time analysis of performance data
  31. 31. #3SPEED
  32. 32. AFFORDABILITY #4
  33. 33. Ready for some pain?
  34. 34. Traditional Performance Testing doesn’t work anymore
  35. 35. 2006
  36. 36. PERFORMANCE TESTINGREVOLUTION
  37. 37. Cloud TestingThe massive resources of the cloud have changed thegame for performance testing. Cloud Testing enablesorganizations to test their consumer facing Web andmobile applications fast, affordably and at scale.There’s now no excuse for the type of Web site disasterswe’ve seen in the past that cost businesses millions inrevenues and perhaps more in reputation. Tom Lounibos CEO SOASTA
  38. 38. What does it bring us?
  39. 39. #1Affordable platform for infinite load
  40. 40. Billions possible points of failureThe new testing challenge
  41. 41. #2Real-time analysis brings wisdom
  42. 42. #3Global and realistic traffic
  43. 43. #4Where is the needle?
  44. 44. In your code
  45. 45. In your infrastructure
  46. 46. Outside your firewall
  47. 47. Extending Performance Beyond the Lab Discovering Issues across teams, time and scale Latency between systems Network Network configuration & Ops Network bandwidth CDN file placement Conflict with other apps DNS routing Load Balancer configurationStage / Team Firewall max capacity Auto-scaling failures Unbalanced web servers Release & Deploy Max sockets exceeded s Global latency variance Security bottlenecks Slow third-party plug-ins Default configuration settings Search technology limits Inadequate server resources Garbage collection Memory leaks Memory leaks Database thread counts Dev & Test Slow pages Inefficient database queries Method-level tuning Test Lab (<10k Vus) Staging (10-100k) Production (> 100k) Scale of Test
  48. 48. CloudTest Live Real-world Issue ResolutionCompany Profile Project ProfileCompany Toy Manufacturer Environment Lab and StagingSize Fortune 500 Application Drupal (Apache, PHP, MySQL) Location Private Hosted Cloud Objective Identify Breaking Points Project Length Five Days (non-consecutive) Lab Staging
  49. 49. CloudTest Live Lab Real-world Issue Resolution•  First test cycle in the lab •  Some servers hitting 100% CPU•  239 hits per second •  Bottleneck in throughput•  40.5 MBit/second bandwidth
  50. 50. CloudTest Live Lab Real-world Issue Resolution•  Second test cycle in the lab •  Some servers hitting 100% CPU•  Higher concurrency target •  Performance problem over time•  Bandwidth stayed the same •  Load balancer flapping servers
  51. 51. CloudTest Live Staging Real-world Issue Resolution•  Bandwidth (238Mbits/s) – ouch!•  CPU at 100%
  52. 52. CloudTest Live Staging Real-world Issue Resolution•  767 Home pages served at 650 megs total transfer.•  That’s an 850kb page weight! Yikes!
  53. 53. CloudTest Live Real-world Issue Resolution •  Poor throughput on the initial tests Lab •  CPU hits 100% •  Bandwidth cap identified in the lab •  Database connections leaking over time •  Load balancer dropping servers Staging •  Massive page weight on the home page •  Application unresponsive at ~2000 users•  Customer happy with the breaking point identified•  Optimized web assets in the home page•  Offloaded assets to the CDN•  Load balancer algorithm optimizations
  54. 54. CloudTest Live Real-world Issue ResolutionCompany Profile Project ProfileCompany Advanced Technology Environment ProductionSize Fortune 50 Application Java (Struts), Oracle Multiple Hosted Private Location Data Centers Objective 172,000 Concurrent Project Length Two Months Production
  55. 55. CloudTest Live Production Real-world Issue ResolutionRecurring error pattern during scenario completion
  56. 56. CloudTest Live Real-world Issue ResolutionAwesome outcome in Production!
  57. 57. CloudTest Live Real-world Issue ResolutionProduction •  Load balancer firmware bug with 2048 bit cert •  Database issues with the login process •  Worked with hardware LB manufacturer on a fix •  Really low error rate •  Successful launch!
  58. 58. Fit for an #5 AGILE WORLD
  59. 59. Real-time collaboration
  60. 60. Continuous performance testing
  61. 61. Cloud Testing helps companies
  62. 62. •  Every day at noon is a holiday spike in traffic.•  Pages must render to the user in under 1 second.•  50% of all revenue from Gilt sales comes in less than 1 hour after sale starts.
  63. 63. Cloud Testing helps dolls
  64. 64. Cloud Testing helps taxes
  65. 65. Cloud Testing helps phones
  66. 66. Testing SOA in the cloudCHALLENGES
  67. 67. It gets crowded
  68. 68. Loose SLAs
  69. 69. Harder to predict
  70. 70. (slightly)Different strategy
  71. 71. Same challenges • Services with no user interface • Data driven • Services outside the organization/cloud • End-to-end scenario • Test ownershipSame Bottom-up performance testing approach • Individual services • Set (integration) of services • End-to-endDeal with Cloud unpredictability • Varying performance encourage continuous performance testing/monitoring. • Infrastructure upgrade could impact performance. • Expect to baseline on premise application and run benchmark in the cloud after migration. • Test the full bottom-up cycle for increased confidence.
  72. 72. Let’s see how this website performs!
  73. 73. Questions/Discussion
  74. 74. The CloudA game changer for web performance testing twitter.com/fredberinger fberinger@soasta.com http://www.fredberinger.com
  75. 75. Photos Credits:Slide 2: http://www.sxc.hu/photo/1219246Slide 18, 19, 20: http://www.sxc.hu/photo/1267744Slide 21: http://www.sxc.hu/photo/1181194Slide 22,23,24: http://www.sxc.hu/photo/396185Slide 25: http://www.flickr.com/photos/tsoumplekas/4495778365/Slide 32: http://www.sxc.hu/photo/1160562Slide 34: http://www.sxc.hu/photo/1331143Slide 46: http://www.sxc.hu/photo/995000Slide 48: http://www.sxc.hu/photo/560250Slide 54: http://www.sxc.hu/photo/561622Slide 59: http://www.flickr.com/photos/caitlinator/4006198703/Slide 60: http://www.sxc.hu/photo/1221951Slide 62: http://www.sxc.hu/photo/1019388
  1. A particular slide catching your eye?

    Clipping is a handy way to collect important slides you want to go back to later.

×