Stress Your Web App Before It
Stresses You: Tools and Techniques
for Extreme Web Testing
Vinicius Senger                  ...
Quote


             “The designer is concerned with what
             happens when 1 user presses a button
              ...
Reality Check
●   Architect designs for 10,000 users
●   Developer programs for 1 user
●   Murphy crashes it on 100 users
...
Goal of This Talk




Understand different methods, techniques and
tools to plan, model, and execute stress tests.




   ...
Agenda

Introduction
Fundamentals of Stress Testing
Tools
Tips
Conclusion
Q&A



                2007 JavaOneSM Conference...
Agenda

Introduction
Fundamentals of Stress Testing
Tools
Tips
Conclusion
Q&A



                2007 JavaOneSM Conference...
Testing Ecosystem
Different tests, different objectives

                                            Test Generation



  ...
Non-Functional Requirements
Different needs, different tests…
   Can my application handle
    5K simultaneous users?    ...
What Is the X
of the Question?



       2007 JavaOneSM Conference | Session TS-9235 |   9
Demand Simulation
X is the number of users…
●   Can my application handle 5K
    simultaneous users?
●   With a given test...
Architecture Decision
     X is the technical architecture…
      ●   Given functional and non-functional requirements,
  ...
Hardware Sizing and
Capacity Planning
X is the hardware (and network)…
●   Given the application and non-functional
    re...
Diagnose Performance Problems
X is the problem
 ●   When a big problem happens and you see
     someone smiling, probably ...
Agenda

Introduction
Fundamentals of Stress Testing
Tools
Tips
Conclusion
Q&A



               2007 JavaOneSM Conference ...
Definition
Functional vs. non-functional requirement testing
●   Functional testing is concerned, most of the
    time, on...
Stress-Testing Phases
Main activities
●   Planning
●   Modelling and designing
●   Construction and instrumentation
●   Ex...
Planning a Stress-Test
Scenarios
●   Stress-test can have different scenarios:
    ●   Credit card companies have differen...
Modelling a Stress-Test
Defining operational profiles

                      User profile/role
Operation             Stand...
Modelling a Stress-Test
Refining input space
●   How do we better represent all possible
    user inputs?
●   Login can be...
Construction
Creating test scripts based on operational profiles
                      User profile / role
Operation
     ...
Agenda

Introduction
Fundamentals of Stress Testing
Tools
Tips
Conclusion
Q&A



                2007 JavaOneSM Conference...
Apache JMeter
●   “De facto” tool for stress testing
    ●   Wide adoption
    ●   Open source (ASL 2.0)
    ●   100% Java...
Essential Elements
●   Test plan: JMeter “project”
●   Workbench: temporary items
●   Thread group: represents demand
●   ...
DEMO
Stressed Hello World




                   2007 JavaOneSM Conference | Session TS-9235 |   24
Other Elements
●   Configuration elements: global configurations
●   Processors: transform request/response data
●   Asser...
DEMO
JavaServer Faces Study Case




                   2007 JavaOneSM Conference | Session TS-9235 |   26
Complementary Tools
      When JMeter is not enough…
      ●    OS monitoring tools
             ●   vmstat, iostat, dtrac...
Agenda

Introduction
Fundamentals of Stress Testing
Tools
Tips
Conclusion
Q&A



                2007 JavaOneSM Conference...
Planning Tips
●   Avoid stress testing only most used scenario
●   Special attention to user definition, when
    talking ...
Planning Tips
Stress-test environment
●   Documentation is paramount:
    ●   OS version and patches
    ●   JVM software
...
JMeter Tips
●   Automatize whatever you can:
    ●   Variables
    ●   Regular expressions
    ●   Processors
    ●   HTTP...
JMeter Tips
●   Beware the interference rule:
    ●   Instrumentation consumes resources
    ●   Test configuration must m...
Agenda

Introduction
Fundamentals of Stress Testing
Tools
Tips
Conclusion
Q&A



                2007 JavaOneSM Conference...
Summary
●   Stress testing is a must
    ●   …Not a might or a should
●   It is not hard to implement
    ●   …Plenty of f...
For More Information
●   JChav: http://jchav.blogspot.com/
●   2005 BOF: http://globalcode.com.br/j1-2005.pdf
●   TS-9646:...
Agenda

Introduction
Fundamentals of Stress Testing
Tools
Tips
Conclusion
Q&A



                2007 JavaOneSM Conference...
Q&A
Vinicius Senger—vinicius@globalcode.com.br
Felipe Leme—felipeal@gmail.com




                     2007 JavaOneSM Conf...
Stress Your Web App Before It
Stresses You: Tools and Techniques
for Extreme Web Testing
Vinicius Senger                  ...
Upcoming SlideShare
Loading in …5
×

Stress Your Web App Before It Stresses You: Tools and Techniques for Extreme Web Testing

2,467 views

Published on

Published in: Technology
2 Comments
9 Likes
Statistics
Notes
No Downloads
Views
Total views
2,467
On SlideShare
0
From Embeds
0
Number of Embeds
8
Actions
Shares
0
Downloads
167
Comments
2
Likes
9
Embeds 0
No embeds

No notes for slide

Stress Your Web App Before It Stresses You: Tools and Techniques for Extreme Web Testing

  1. 1. Stress Your Web App Before It Stresses You: Tools and Techniques for Extreme Web Testing Vinicius Senger Felipe Leme Architect and Instructor Architect Globalcode Matera Systems http://www.globalcode.com.br http://matera.com TS-9235 2007 JavaOneSM Conference | Session TS-9235 |
  2. 2. Quote “The designer is concerned with what happens when 1 user presses a button and the architect is concerned with what happens when 10,000 users press a button.” Sun Certified Enterprise Architect for J2EE Technology Study Guide. Page 6. Mark Cade, Simon Roberts. 2007 JavaOneSM Conference | Session TS-9235 | 2
  3. 3. Reality Check ● Architect designs for 10,000 users ● Developer programs for 1 user ● Murphy crashes it on 100 users ● Stress testing to the rescue! 2007 JavaOneSM Conference | Session TS-9235 | 3
  4. 4. Goal of This Talk Understand different methods, techniques and tools to plan, model, and execute stress tests. 2007 JavaOneSM Conference | Session TS-9235 | 4
  5. 5. Agenda Introduction Fundamentals of Stress Testing Tools Tips Conclusion Q&A 2007 JavaOneSM Conference | Session TS-9235 | 5
  6. 6. Agenda Introduction Fundamentals of Stress Testing Tools Tips Conclusion Q&A 2007 JavaOneSM Conference | Session TS-9235 | 6
  7. 7. Testing Ecosystem Different tests, different objectives Test Generation Requirement Based Code Based Non-Functional Functional Coverage Based Performance Scalability 2007 JavaOneSM Conference | Session TS-9235 | 7
  8. 8. Non-Functional Requirements Different needs, different tests…  Can my application handle 5K simultaneous users? ● Simultaneous users  Which architecture should ● Continuous usage we use? ● Hardware sizing  My server crashes everyday… ● Capacity planning  Will our server survive the next 2 years? ● Architecture decisions  Will application work after 4 ● Stress-test driven development months of continuous usage? ● Continuous performance  What server should we buy? ● Find a bottleneck/diagnose  We need to afford 20K user performance problem 2007 JavaOneSM Conference | Session TS-9235 | 8
  9. 9. What Is the X of the Question? 2007 JavaOneSM Conference | Session TS-9235 | 9
  10. 10. Demand Simulation X is the number of users… ● Can my application handle 5K simultaneous users? ● With a given test configuration (data center environment) and Java™ Platform, Enterprise Edition (Java™ EE platform) Web App, how many users can it afford? app.war + = X 2007 JavaOneSM Conference | Session TS-9235 | 10
  11. 11. Architecture Decision X is the technical architecture… ● Given functional and non-functional requirements, which architecture should we use? ● Should we cluster Web + Enterprise JavaBeans™ (EJB™) technology container together? JMS API JavaServer™ Faces AJAX X Hibernate + 5K users = WebService Aspect Patterns EJB Swing Architecture JMS = Java Message Service JCA JCA = J2EE™ Connector Architecture 2007 JavaOneSM Conference | Session TS-9235 | 11
  12. 12. Hardware Sizing and Capacity Planning X is the hardware (and network)… ● Given the application and non-functional requirements, how much hardware do we need? app.war + 5K users = X 2007 JavaOneSM Conference | Session TS-9235 | 12
  13. 13. Diagnose Performance Problems X is the problem ● When a big problem happens and you see someone smiling, probably they already found a someone to blame! Java™ DataBase EJB Specification Pool? Connectivity (JDBC™) JDBC software Pool? + app.war = Dead-lock? Memory? 5K users O.S.? AJAX? SQL code? Framework? Network? Murphy’s Law? Garbage collector? 2007 JavaOneSM Conference | Session TS-9235 | 13
  14. 14. Agenda Introduction Fundamentals of Stress Testing Tools Tips Conclusion Q&A 2007 JavaOneSM Conference | Session TS-9235 | 14
  15. 15. Definition Functional vs. non-functional requirement testing ● Functional testing is concerned, most of the time, on single-thread and single-user testing ● Stress-testing is concerned on multi-thread /multi-user testing ● Repetition, concurrency, magnitude, and random variation are parts of a stress-test 2007 JavaOneSM Conference | Session TS-9235 | 15
  16. 16. Stress-Testing Phases Main activities ● Planning ● Modelling and designing ● Construction and instrumentation ● Execution ● Data analyses 2007 JavaOneSM Conference | Session TS-9235 | 16
  17. 17. Planning a Stress-Test Scenarios ● Stress-test can have different scenarios: ● Credit card companies have different demands during the year ● E-commerce demand changes during promotions ● Financial market is susceptible to external factors (wars, elections, rumors) ● Different scenarios, different load factor ● The more you plan, the more you cover! 2007 JavaOneSM Conference | Session TS-9235 | 17
  18. 18. Modelling a Stress-Test Defining operational profiles User profile/role Operation Standard Investor Manager Corporate (20%) (20%) (10%) (50%) Withdraw 20% 5% 25% Deposit 15% 15% 25% Transfer 15% 15% 30% Account report 50% 20% 20% 20% Create an account 20% Inactive an account 30% Make investments 45% 30% 2007 JavaOneSM Conference | Session TS-9235 | 18
  19. 19. Modelling a Stress-Test Refining input space ● How do we better represent all possible user inputs? ● Login can be a string: ● Minimum size is 4 ● Maximum size is 10 ● Only numbers and letters allowed ● Test input space for username: ● Ana, A1 ● Ana1, James, JonhLennon, JonhLennon12 2007 JavaOneSM Conference | Session TS-9235 | 19
  20. 20. Construction Creating test scripts based on operational profiles User profile / role Operation Standard Investor Manager Corporate (20%) (20%) (10%) (50%) Withdraw 20% 5% 25% Deposit 15% 15% 25% Transfer 15% 15% 30% Account report 50% 20% 20% 20% Create an account 20% Inactive an account 30% Make investments 45% 30% 2007 JavaOneSM Conference | Session TS-9235 | 20
  21. 21. Agenda Introduction Fundamentals of Stress Testing Tools Tips Conclusion Q&A 2007 JavaOneSM Conference | Session TS-9235 | 21
  22. 22. Apache JMeter ● “De facto” tool for stress testing ● Wide adoption ● Open source (ASL 2.0) ● 100% Java technology Desktop Application ● Very mature (current release: 2.2) ● Originally intended for testing web (HTTP requests) ● Flexible design: Supports JMS technology, POP3, TCP, JUnit, SOAP, FTP, LDAP, JDBC software, and many other requests 2007 JavaOneSM Conference | Session TS-9235 | 22
  23. 23. Essential Elements ● Test plan: JMeter “project” ● Workbench: temporary items ● Thread group: represents demand ● Samplers: protocol-specific requests ● Listeners: handle results ● Minimum usage: ● Thread Group (X simultaneous users) ● A request ● One or more listener 2007 JavaOneSM Conference | Session TS-9235 | 23
  24. 24. DEMO Stressed Hello World 2007 JavaOneSM Conference | Session TS-9235 | 24
  25. 25. Other Elements ● Configuration elements: global configurations ● Processors: transform request/response data ● Assertions: test validity of responses ● Timers: modify the tempo between requests ● Logic controllers: groups elements according to logical conditions ● HTTP Proxy Server: add elements by mimicking real usage 2007 JavaOneSM Conference | Session TS-9235 | 25
  26. 26. DEMO JavaServer Faces Study Case 2007 JavaOneSM Conference | Session TS-9235 | 26
  27. 27. Complementary Tools When JMeter is not enough… ● OS monitoring tools ● vmstat, iostat, dtrace ● Java Virtual Machine (JVM™) monitoring ● Java Management Extensions (JMX™) technology console ● Profiler Integration ● Build integration ● Apache Ant tasks ● JChav The terms “Java Virtual Machine” and “JVM” mean a Virtual Machine for the Java™ platform. 2007 JavaOneSM Conference | Session TS-9235 | 27
  28. 28. Agenda Introduction Fundamentals of Stress Testing Tools Tips Conclusion Q&A 2007 JavaOneSM Conference | Session TS-9235 | 28
  29. 29. Planning Tips ● Avoid stress testing only most used scenario ● Special attention to user definition, when talking about simultaneous users stress-testing ● Usage of real input data space instead of synthetic data (whenever possible) 2007 JavaOneSM Conference | Session TS-9235 | 29
  30. 30. Planning Tips Stress-test environment ● Documentation is paramount: ● OS version and patches ● JVM software ● Database/connection pool ● Full hardware description ● Techniques for monitoring ● Input space ● Definition of user profiles 2007 JavaOneSM Conference | Session TS-9235 | 30
  31. 31. JMeter Tips ● Automatize whatever you can: ● Variables ● Regular expressions ● Processors ● HTTP Request Default ● Remember: HTTP is stateless ● Cookies and Session ID ● JavaServer Faces state ● Solution: HTTP Cookie Manager 2007 JavaOneSM Conference | Session TS-9235 | 31
  32. 32. JMeter Tips ● Beware the interference rule: ● Instrumentation consumes resources ● Test configuration must mirror real world ● Servers, network, OS, load ● JMeter proxy is your friend ● Powerful and handy tool ● Ajax requests ● Session-based workflow ● JavaServer Faces requests 2007 JavaOneSM Conference | Session TS-9235 | 32
  33. 33. Agenda Introduction Fundamentals of Stress Testing Tools Tips Conclusion Q&A 2007 JavaOneSM Conference | Session TS-9235 | 33
  34. 34. Summary ● Stress testing is a must ● …Not a might or a should ● It is not hard to implement ● …Plenty of free tools available ● Plan earlier, test always ● …Do not wait until it is too late! 2007 JavaOneSM Conference | Session TS-9235 | 34
  35. 35. For More Information ● JChav: http://jchav.blogspot.com/ ● 2005 BOF: http://globalcode.com.br/j1-2005.pdf ● TS-9646: Performance-Tune Your Ajax Application 2007 JavaOneSM Conference | Session TS-9235 | 35
  36. 36. Agenda Introduction Fundamentals of Stress Testing Tools Tips Conclusion Q&A 2007 JavaOneSM Conference | Session TS-9235 | 36
  37. 37. Q&A Vinicius Senger—vinicius@globalcode.com.br Felipe Leme—felipeal@gmail.com 2007 JavaOneSM Conference | Session TS-9235 | 37
  38. 38. Stress Your Web App Before It Stresses You: Tools and Techniques for Extreme Web Testing Vinicius Senger Felipe Leme Architect and Instructor Architect Globalcode Matera Systems http://www.globalcode.com.br http://matera.com TS-9235 2007 JavaOneSM Conference | Session TS-9235 |

×