Web Performance Analysis - TCF Pro 2009


Published on

A discussion of web performance analysis and testing.

Published in: Technology
  • Be the first to comment

  • Be the first to like this

No Downloads
Total views
On SlideShare
From Embeds
Number of Embeds
Embeds 0
No embeds

No notes for slide

Web Performance Analysis - TCF Pro 2009

  1. 1. Sur vey of Web Performance Analysis Guy Ferraiolo, Norstrilia Soft ware, Inc. gdf@norstriliasoft ware.com copyright 2009, Guy Ferraiolo some rights reser ved This work is licensed under the Creative Commons Attribution-No Derivative Works 3.0 License. To view a copy of this license, visit http://creativecommons.org/licenses/by-nd/3.0/ or send a letter to Creative Commons, 171 Second Street, Suite 300, San Francisco, California, 94105, USA.
  2. 2. Introduction
  3. 3. Who am I? Consultant, Principal Soft ware Engineer, 8 years at a major internet presence, 30 years in the soft ware industry Ran performance testing effort, t wo sophisticated performance test labs Committer and contributor to Apache Soft ware Foundation
  4. 4. My Environment In a performance group Work with engineering organizations to facilitate and empower their performance analysis efforts Responsibilities clearly divided among organizations tiered architecture, mostly
  5. 5. Approach of This Talk Find a conceptual basis the task is complex Provide some advice based on experience
  6. 6. Focus of This Talk Sur vey only, much more information is available Primarily concepts, some discussion of tools Heuristics and work processes General, using web examples, HTTP
  7. 7. Will Provide Definite advice Amusing anecdotes Some tool recommendations
  8. 8. Won’t Provide Magic bullets Simple answers
  9. 9. Performance Analysis
  10. 10. Goals Know what performance results are Understand reasons for performance results Save money Save time
  11. 11. Two Different Web Performance Aspects Page delivery time Resource cost to deliver a page Not entirely unrelated but commonly confused
  12. 12. Why Performance Matters Poor performance means less traffic and therefore less revenue This relationship is surprizingly strong, perhaps 20% less traffic for an additional 0.1 seconds Infrastructure is very expensive
  13. 13. Why Resource Cost Matters A highly dynamic page may be very light weight but require enormous effort to assemble - the example is Google Even if a page is fast to deliver the cost of the infrastructure to create it can be a major business problem
  14. 14. Performance Analysis is primarily analytic but testing is essential works best when integrated into the development process
  15. 15. Performance Analysis Consists of Detecting Performance Issues some sort of monitoring is needed Localizing Performance Issues usually done with analysis Resolving Performance Issues often easy but testing can be critical
  16. 16. Performance Analysis is not code monkey work is not fast is not highly rigorous
  17. 17. Types of Performance Analysis Capacity planning Acceptance testing Design and algorithm selection
  18. 18. Good & Bad News Meaningful results can be obtained Results sometimes not expected Requires a serious investment Is inherently iterative
  19. 19. Where to Start A hierarchy of concepts - most important policies - derived from concepts tools - necessary but not dominant
  20. 20. Concepts
  21. 21. The Big Concept Reduction of Uncertainty overarching concept provides answers to why?, what? and do I care?
  22. 22. How to Apply the Big Concept Relate performance to business needs provide enough information to allow management to make meaningful business decisions gain enough information to allow extrapolation of trends
  23. 23. Other Concepts Intuition is likely to be wrong The process is inherently iterative business process technical process An early start is very helpful
  24. 24. Policies
  25. 25. Policies Analyze and test enough to make a business decision Have a clearly defined process Define your purpose Make a quantitative estimate of the traffic Let those who know something, do it
  26. 26. More Policies Communicate openly and often Compartmentalize responsibility Do not use performance testing to expose functional problems, this is not QA (quality assurance)
  27. 27. Processes
  28. 28. Process Attributes Defined Documented Public Includes pre and post testing
  29. 29. Pre-Test Process Assign responsibilities, be definite about who Declare goals Record configuration Assess traffic quantity and type
  30. 30. Post-Test Process Meet definite end criteria Report results Make recommendations Communicate issues
  31. 31. Test Process Dynamic, goal seeking Interactive Iterative Duration depends on behavior
  32. 32. Test Options Live or lab End-to-end or component
  33. 33. The Most Common Error Unjustified belief that the problem is code related, usually architecture, data, traffic
  34. 34. Common Pitfalls Unclear goals N-dimensional matrix of tests Emphasis on the tool Emphasis on the code
  35. 35. More Common Pitfalls Focus on a single tool for testing Belief in a single answer to the problem
  36. 36. Tools
  37. 37. Tools Issues Open source or proprietary? Proxy capture of user actions? Scriptable? Randomizable? Enforce policies?
  38. 38. Two Types of Tools Load generators Test environment managers
  39. 39. Load Generators flood JMeter many others
  40. 40. Load Generator Issues Synchronous Randomizable Script or proxy Load ramping
  41. 41. Load Generator Obsession What people usually mean when they talk about performance testing tools Necessary Far from sufficient There are many
  42. 42. Environment Managers Unlike load generators, few if any exist But, you can get lots of what is needed through policies and procedures
  43. 43. Test Environment Controlled Isolated Automated Recording
  44. 44. Test Facility Components Manager Load generator System under test Data recorder Report generator
  45. 45. Performance Tests
  46. 46. Test Design Much more in common with experiment design than statistical analysis Statistics are good, but ... You don’t need statistics to know which tire is flat Good test design can make it that obvious
  47. 47. Test Components Two parts load side system under test side
  48. 48. Test Results Also have t wo sides Load side requests/s, s/request timing of parts of the request System under test side system and application metrics
  49. 49. Test Analysis Reality check results and test parameters Compare to actual or derive a baseline Follow good analytic methodology multiple tests, etc.
  50. 50. Pet Peeves Load ramping if you want different loads, run different tests actual load changes hard to match Proxy capture better to get real traffic from logs
  51. 51. Wrap Up
  52. 52. Key Points Strong concepts Well-defined process Experiment design Iterative
  53. 53. How to Solve Your Problem I’m available for consulting Introductory talks for Managers and Engineers tools, instruction, procedures, policies
  54. 54. Further Info - General Information http:/opensourcetesting.org/ performance.php http:/www.soft wareqatest.com/ qat web1.html/#LOAD search for ‘web performance testing’
  55. 55. Further Info - Proprietary TestRunner OpenLoad
  56. 56. Further Info - Load Generators JMeter http://jakarta.apache.org/jmeter/ Flood http://httpd.apache.org/test/flood/
  57. 57. Further Info - Environment Managers OpenSTA http://opensta.org Faban http://faban.sunsource.net The Grinder http://grinder.sourceforge.net
  58. 58. Excellent Book Programming Pearls, Jon Bentley
  59. 59. Thanks!
  60. 60. Sur vey of Web Performance Analysis Guy Ferraiolo, Norstrilia Soft ware, Inc. gdf@norstriliasoft ware.com
  61. 61. contact information