Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.
WebPerfDays NY 2014 
Tools of the Trade: Load Testing 
Alexander Podelko 
alex.podelko@oracle.com 
alexanderpodelko.com/bl...
About Me 
• Specialized in performance for the last 17 years 
• Currently performance testing and optimization 
of Hyperio...
Front- vs. Back-End 
• Back-end response times depend on load 
Thanks to Andy Hawkes for the graph 
3
Load Testing 
Applying multi-user synthetic load to the system 
– Load testing 
– Performance testing 
– Stress testing 
–...
Can System Handle Peak Load? 
• You can’t know without testing: 
5
Verify Multi-User Performance 
• Single-user improvement may lead to multi-user 
performance degradation 
6
What Else Load Testing Adds 
• Performance optimization 
– Apply exactly the same load 
– See if the change makes a differ...
Commercial Tools 
Gartner Magic Quadrant for Quality Suites: 
• Leaders: 
– HP (LoadRunner) 
– IBM (Rational Performance T...
Open Source Tools 
• 53 tools listed at 
http://www.opensourcetesting.org/performance.php 
• Jmeter 
– Clear leader in pop...
Mixed Business Models 
• Advanced functionality 
– With small tests for free 
• Freemium models 
– LoadRunner, NeoLoad (up...
Load Testing Tools 
Differ drastically in: 
– Supported approaches / protocols 
– Functionality / extendibility 
– Support...
Record and Playback: Protocol 
Level 
Load Generator Server 
Load Testing Tool 
Virtual Users 
Application 
Network 
12
Record and Playback: UI Level 
13 
Load Generator Server 
Load Testing Tool 
Virtual 
Users 
Application 
Network 
Brows 
...
Programming 
Load Generator Server 
Load Testing Tool App. 
Virtual 
Users 
Application 
Network 
API 
14
Environments 
• Lab vs. Service (SaaS) vs. Cloud (IaaS) 
– For both the system and load generators 
• Test vs. Production ...
Scenarios 
• System validation for high load 
– Outside load (service or cloud), production system 
– Wider scope, lower r...
Integration 
• Reporting and Analysis 
• Monitoring / APM 
– System 
– Application insights (APM) 
• AppDynamics, New Reli...
Tool’s Ecosphere 
18 
Name Number of found 
documents by 
Google 
Number of found 
US positions at 
Monster.com 
[HP] Load...
Summary 
• There is no best tool – it depends on your 
needs 
• Almost every tool works for a small and simple 
web site 
...
Questions? 
Alexander Podelko 
alex.podelko@oracle.com 
alexanderpodelko.com/blog 
@apodelko 
Need More? Come to the Perfo...
Upcoming SlideShare
Loading in …5
×

Tools of the Trade: Load Testing - Ignite session at WebPerfDays NY 14

1,313 views

Published on

Tools of the Trade: Load Testing - an Ignite session at WebPerfDays NY 2014. Some consideration about load testing and selecting load testing tools - as much as could be squeezed into 5 min / 20 slides.

Published in: Technology
  • Be the first to comment

Tools of the Trade: Load Testing - Ignite session at WebPerfDays NY 14

  1. 1. WebPerfDays NY 2014 Tools of the Trade: Load Testing Alexander Podelko alex.podelko@oracle.com alexanderpodelko.com/blog @apodelko September 18, 2014
  2. 2. About Me • Specialized in performance for the last 17 years • Currently performance testing and optimization of Hyperion products at Oracle • Board director at CMG http://cmg.org – Organization of performance and capacity professionals – Conference November 3-6, 2014 in Atlanta, GA Disclaimer: The views expressed here are my personal views only and do not necessarily represent those of my current or previous employers. All brands and trademarks mentioned are the property of their owners. 2
  3. 3. Front- vs. Back-End • Back-end response times depend on load Thanks to Andy Hawkes for the graph 3
  4. 4. Load Testing Applying multi-user synthetic load to the system – Load testing – Performance testing – Stress testing – Scalability testing – Volume testing – Reliability testing – Concurrency testing – Realistic Testing – Endurance testing – Longevity testing – Soak testing – Stability testing 4
  5. 5. Can System Handle Peak Load? • You can’t know without testing: 5
  6. 6. Verify Multi-User Performance • Single-user improvement may lead to multi-user performance degradation 6
  7. 7. What Else Load Testing Adds • Performance optimization – Apply exactly the same load – See if the change makes a difference • Debugging/verification of multi-user issues • Testing self-regulation functionality – Such as auto-scaling or changing the level of service depending on load 7
  8. 8. Commercial Tools Gartner Magic Quadrant for Quality Suites: • Leaders: – HP (LoadRunner) – IBM (Rational Performance Testing) – Microsoft (Visual Studio Web Performance…) – Oracle (Application Testing Suite) – SOASTA(CloudTest) – Borland (SilkPerformer) • Peformance: Neotys (NeoLoad), Blazemeter 8
  9. 9. Open Source Tools • 53 tools listed at http://www.opensourcetesting.org/performance.php • Jmeter – Clear leader in popularity • Gatling – Getting popularity recently • Grinder, OpenSTA – Were popular/looks like no development recently 9
  10. 10. Mixed Business Models • Advanced functionality – With small tests for free • Freemium models – LoadRunner, NeoLoad (up 50 users free) – CloudTest (Lite up to 100 users free) • BlazeMeter service using open source JMeter • Important when more people get involved – Continuous Integration, Agile development 10
  11. 11. Load Testing Tools Differ drastically in: – Supported approaches / protocols – Functionality / extendibility – Supported environments – Scalability – Result analysis – Integration (monitoring, APM, CI, etc.) – Cost/Licensing – Available support and skills 11
  12. 12. Record and Playback: Protocol Level Load Generator Server Load Testing Tool Virtual Users Application Network 12
  13. 13. Record and Playback: UI Level 13 Load Generator Server Load Testing Tool Virtual Users Application Network Brows ers
  14. 14. Programming Load Generator Server Load Testing Tool App. Virtual Users Application Network API 14
  15. 15. Environments • Lab vs. Service (SaaS) vs. Cloud (IaaS) – For both the system and load generators • Test vs. Production • No best solution, depends on your goals / system 15
  16. 16. Scenarios • System validation for high load – Outside load (service or cloud), production system – Wider scope, lower repeatability • Performance optimization / troubleshooting – Isolated lab environment – Limited scope, high repeatability • Testing in Cloud – Lowering costs (in case of periodic tests) – Limited scope, low repeatability 16
  17. 17. Integration • Reporting and Analysis • Monitoring / APM – System – Application insights (APM) • AppDynamics, New Relic, Dynatrace, etc. • Continuous Integration – CI servers (Jenkins, Hudson, etc.) – Adding to automatic build process 17
  18. 18. Tool’s Ecosphere 18 Name Number of found documents by Google Number of found US positions at Monster.com [HP] LoadRunner 894,000 170 [Apache] JMeter 688,000 90 [Borland] SilkPerformer 138,000 12 [Neotys] NeoLoad 87,100 3 [SOASTA] CloudTest 51,500 2 LoadStorm 18,000 -
  19. 19. Summary • There is no best tool – it depends on your needs • Almost every tool works for a small and simple web site • If using more sophisticated technologies, always check if the tool / approach supports it 19
  20. 20. Questions? Alexander Podelko alex.podelko@oracle.com alexanderpodelko.com/blog @apodelko Need More? Come to the Performance and Capacity conference http://www.cmg.org • November 3-6, 2014 in Atlanta, GA • Perfect complement to Velocity / WebPerfDays

×