Adding Performance Testing to a Software Development Project

1,644 views

Published on

Presented at Jasig 2010 conference March, 2010 in San Diego.

Published in: Technology
0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total views
1,644
On SlideShare
0
From Embeds
0
Number of Embeds
13
Actions
Shares
0
Downloads
0
Comments
0
Likes
0
Embeds 0
No embeds

No notes for slide

Adding Performance Testing to a Software Development Project

  1. 1. Adding Performance Testing to a Software Development Project Cris J. Holdorph Lennard Fuller © Copyright Unicon, Inc., 2008. Some rights reserved. This work is licensed under a Creative Commons Attribution-Noncommercial-Share Alike 3.0 United States License. To view a copy of this license, visit http://creativecommons.org/licenses/by-nc-sa/3.0/us/
  2. 2. Agenda 1. What is performance testing? 2. Performance Test Environment 3. Setting Goals 4. Performance Test Tools 5. Performance Test Data 6. Analyzing the Results 7. Optimizing the System 2
  3. 3. What is performance testing? 3
  4. 4. Performance Testing Overview Purpose of performance testing is NOT to find bugs. ● Intended to eliminate performance bottlenecks ● Establish a baseline for future regression testing ● Results are compared to a clearly defined set of Goals 4
  5. 5. Typical Performance Testing Cycle Run Test Optimize Analyze * * Repeat until Goals are reached. 5
  6. 6. Performance Test Environment 6
  7. 7. Performance Test Environment ● Separate Hardware – Production duplicate if possible ● External Systems – Will your performance test system hit your production LDAP? SSO? SIS? 7
  8. 8. Goals ● Expected Users ● Expected Response times ● Making sense of the misunderstood – User Concurrency ● Time to return HTML for a page is not equal to the time to load a page – Testing Full Page loads versus HTML only page loads 8
  9. 9. Tools ● Grinder ● JMeter ● Commercial – Loadrunner – Silk Performer ● Recording ● Modifying / Tokenizing 9
  10. 10. Prepping the Test ● Set up Test Environment ● Load Database with Realistic Data – Type of data – Amount of data ● Record the Test Script – Record (including sleep time) – Tokenize / Modify / Edit ● Set up Test Scenario – Test script mix – Sleep time randomization 10 – Ramp up time
  11. 11. Running the Test ● Gather information during the test – CPU, IO, memory, GC ● Zero out logs before test begins ● Reset the database before test begins – Only necessary if performance test changes data in certain ways, or if test adds data ● Restart the application ● Prime the application – Login as a test user(s) and do actions similar to what the performance test does ● Pressing run (start) on the test tool 11
  12. 12. Analysis ● Gather statistics from different sources – Test tool, sar, custom shell script, gc logs ● Examine application logs – Are there any errors or stack traces that show a performance related failure? ● uPortal only – uPortal built in stats database not recommended for performance testing, but can help debug production performance problems ● Look for Trends ● Look for Spikes 12
  13. 13. Optimization Can be difficult, often more of an art than a science due to complexity. Cardinal Rule of Performance Optimization: “Modify ONE variable at a time and remeasure.” 13
  14. 14. Some Areas for Optimization ● JVM args ● Application Content – Portlets, Layout, etc. ● Application Configuration – Logging Level, Logging Location – Database Connection Pooling – Tune Cache Configuration ● System Architecture – Use Apache HTTPD in front of Tomcat to serve static resources – Use hardware load balancing / SSL 14 – Increase available network bandwidth
  15. 15. Part of Normal Development ● How to make Performance Testing part of the normal Software Development lifecycle ● Benefits – Regression Testing (find problems before they occur in production) – Anticipate Hardware needs before code rolls to production ● Need – Test Environment – Test Scripts – People to run/analyze the test 15
  16. 16. Limitations: Spikes and Valleys ● Performance testing does NOT simulate the real world – Both Queueing Theory and Experience show that usage is not uniform! – Caution: Outside factors could influence your system 16
  17. 17. Real world looks more like... 17
  18. 18. Limitations ● Users don't all do the same things in the same order ● Real usage can differ wildly in terms of associations, and actual data – i.e. chinese class with 400 students ● AJAX / client side activities are hard to performance test, using traditional performance testing tools 18
  19. 19. Questions & Answers Cris Holdorph Lennard Fuller Unicon, Inc. holdorph@unicon.net lfuller@unicon.net www.unicon.net 19

×