Ajax Load Testing Considerations<br />Load Testing Ajax Websites<br />
Background	<br />Eric Beland <br />Co-Founder, Testomatix<br />http://testomatix.com<br />ebeland@testomatix.com<br />Twit...
Testing Ajax Websites<br />
Will it Scale?<br />What happens when people show up?<br />
Testing at the HTTP Layer<br />
Testing at the HTTP Layer<br />Advantage: Scalable - No browser<br />Small memory and CPU footprint  <br />Tools:  HP Load...
Challenges At the HTTP layer<br />Things that aren’t as smooth when simulating the browser…<br />
Hard-Coded URLs<br />
Hard-coded URLs<br />If a request is dependent on a previous Ajax request which, after a certain number of users starts fa...
Recording Issues – Is it recording?<br />Does your tool record AJAX traffic?<br />VS 2005 doesn&apos;t record Ajax calls.<...
Things that won’t work, at least easily<br />Highly JavaScript-dependent code.<br />Any client-server interaction that is ...
Recording Issues – Ajax+ REST Issues<br />REST (Representational State Transfer) makes use of 4 HTTP verbs <br />GET, POST...
Script Maintenance<br />Script recording<br />	Unless you remember, or your testers determine every place Ajax has been ad...
Statefulness Consideration<br />Ajax requests may require a session cookie, or value in a cookie.<br />If the cookie is no...
Client-side JS Timestamp on Cache<br />Timestampsmay be hard coded in your recorded posts<br />We have seen JavaScript cac...
Validate Ajax Returns<br />For valid testing, the return values of AJAX calls must be validated.<br />If they are not vali...
AutoComplete Fields<br />Think time between Ajax requests—users type quickly.<br />In order to replicate real load, and ex...
Ajax Autocomplete<br />Autocomplete made these requests:<br /><ul><li>Parameterization is tricky.
The correct way to generate the load is to make multiple requests for the parameter.
Additionally, it looks like cp=could be a character count. </li></ul>http://clients1.google.com/complete/search?hl=en&clie...
AJAX Polling or AJAX Streaming<br />JavaScript which does AJAX polling in the background. <br />Polling interval may chang...
Reporting<br />Many tools work on the “page” model for load testing.  <br />Ajax apps do not necessarily follow the page m...
AJAX Response Times<br />AJAX response time expectations<br />Not the same as page load response expectations<br />Ajax re...
Timeouts<br />Choosing timeouts<br />Load/Scalability tests<br />Lower timeouts are appropriate for Ajax heavy application...
AJAX/UI response time rules <br />Excerpt from Jacob Nielsen&apos;s Usability Engineering<br />&quot;The basic advice rega...
New Alternatives to HTTP Layer Testing<br />Browser-based load testing.<br />Historically, not very scalable.<br />Enter t...
Benefits<br />Script maintainability<br />Changes to the websites AJAX code will flow through as JavaScript is executed at...
Upcoming SlideShare
Loading in...5
×

Ajax Load Testing Considerations

947

Published on

These slides are from a presentation given at "The Ajax Experience" in Boston. The presentation covers common Ajax load testing gotchas, as well as an overview of testing approaches for Ajax applications.

Testomatix, provides full service web application load testing on a consultative basis.

http://testomatix.com

Published in: Technology
0 Comments
1 Like
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total Views
947
On Slideshare
0
From Embeds
0
Number of Embeds
1
Actions
Shares
0
Downloads
36
Comments
0
Likes
1
Embeds 0
No embeds

No notes for slide

Ajax Load Testing Considerations

  1. 1. Ajax Load Testing Considerations<br />Load Testing Ajax Websites<br />
  2. 2. Background <br />Eric Beland <br />Co-Founder, Testomatix<br />http://testomatix.com<br />ebeland@testomatix.com<br />Twitter: @testomatix<br /> Testomatix provides full service performance testing including expert planning, scripting, and test execution. We test for you, and help you tune. <br />
  3. 3. Testing Ajax Websites<br />
  4. 4. Will it Scale?<br />What happens when people show up?<br />
  5. 5. Testing at the HTTP Layer<br />
  6. 6. Testing at the HTTP Layer<br />Advantage: Scalable - No browser<br />Small memory and CPU footprint <br />Tools: HP LoadRunner, Jmeter, Grinder, Oracle e-Load<br />Created by: <br />Recording from within the browser—browser plugins or hooks.<br />Intercepting requests with a proxy tool. <br />May be hand-scripted.<br />
  7. 7. Challenges At the HTTP layer<br />Things that aren’t as smooth when simulating the browser…<br />
  8. 8. Hard-Coded URLs<br />
  9. 9. Hard-coded URLs<br />If a request is dependent on a previous Ajax request which, after a certain number of users starts failing, a real user would fail. <br />If you have a hard-coded the URL, your test will pass when it shouldn’t.<br />If an Ajax request is vital to the user interaction,<br />Content-checked (match against text in the HTML)<br />Or defined as a RegExp source for navigation<br />
  10. 10.
  11. 11. Recording Issues – Is it recording?<br />Does your tool record AJAX traffic?<br />VS 2005 doesn&apos;t record Ajax calls.<br />Proxy-based tools tend to do well when recording Ajax.<br />Some proxy based recorders can’t record https traffic. <br />Jmeter, for example<br />
  12. 12. Things that won’t work, at least easily<br />Highly JavaScript-dependent code.<br />Any client-server interaction that is dependent on complex client-side computation<br />Unless you re-implement the logic in your testing tool…<br />Client-side decompression<br />Client side JS encryption (Don’t do this. Really.)<br />Obfuscation<br />Streaming<br />
  13. 13. Recording Issues – Ajax+ REST Issues<br />REST (Representational State Transfer) makes use of 4 HTTP verbs <br />GET, POST, PUT and DELETE<br />Browsers do not have support for PUT and DELETE currently (HTML 5) <br />But it IS supported in the XMLHttpRequest implementation in all major browsers.<br />But, your load testing tool may not be able to record it, or replay it.<br />Currently (Sept. 09) the case in Oracle e-Load. <br />Used to be an issue in JMeter.<br />Likely still an issue in other tools.<br />
  14. 14. Script Maintenance<br />Script recording<br /> Unless you remember, or your testers determine every place Ajax has been added or removed from a script, you need to recreate your scripts for them to be accurate.<br />A “real life” example:<br />http://thedailywtf.com/Articles/Slowing-Time.aspx<br />
  15. 15. Statefulness Consideration<br />Ajax requests may require a session cookie, or value in a cookie.<br />If the cookie is not configured to be updated, the request will fail.<br />
  16. 16. Client-side JS Timestamp on Cache<br />Timestampsmay be hard coded in your recorded posts<br />We have seen JavaScript caching, with timestamps which need to be updated.<br />
  17. 17. Validate Ajax Returns<br />For valid testing, the return values of AJAX calls must be validated.<br />If they are not validated, the script may appear to work past the number of users which the application can support.<br />
  18. 18. AutoComplete Fields<br />Think time between Ajax requests—users type quickly.<br />In order to replicate real load, and exercise any caching effectively, requests must be parameterized.<br />AutoComplete fields send requests key-by-key. <br />Parameterization may not account for this properly.<br />
  19. 19.
  20. 20. Ajax Autocomplete<br />Autocomplete made these requests:<br /><ul><li>Parameterization is tricky.
  21. 21. The correct way to generate the load is to make multiple requests for the parameter.
  22. 22. Additionally, it looks like cp=could be a character count. </li></ul>http://clients1.google.com/complete/search?hl=en&client=hp&q=a&cp=1<br />http://clients1.google.com/complete/search?hl=en&client=hp&q=aja&cp=3<br />http://clients1.google.com/complete/search?hl=en&client=hp&q=ajax&cp=4<br />
  23. 23. AJAX Polling or AJAX Streaming<br />JavaScript which does AJAX polling in the background. <br />Polling interval may change, which changes the load associated with a page.<br />Comet (Streaming)<br />When added or removed, load scripts must be updated.<br />
  24. 24. Reporting<br />Many tools work on the “page” model for load testing. <br />Ajax apps do not necessarily follow the page model, and in many cases, the application is not “working” when the AJAX functionality is not working.<br />Ajax timeouts must be treated as full-fledged failures.<br />
  25. 25. AJAX Response Times<br />AJAX response time expectations<br />Not the same as page load response expectations<br />Ajax response times should follow UI response time rules, which have their own laws, and generally need to be faster.<br />A 6 second page load might be ok, but a 6 second lag between UI responses is painful<br />UI Responsiveness rules are pretty brutal when applied to web applications.<br />
  26. 26. Timeouts<br />Choosing timeouts<br />Load/Scalability tests<br />Lower timeouts are appropriate for Ajax heavy applications.<br />Determine how many users you can support within acceptable-response times.<br />10 seconds is probably a reasonable threshold for an Ajax request. <br />
  27. 27. AJAX/UI response time rules <br />Excerpt from Jacob Nielsen&apos;s Usability Engineering<br />&quot;The basic advice regarding response times has been about the same for almost thirty years [Miller 1968; Card et al. 1991]: <br />0.1 second is about the limit for having the user feel that the system is reacting instantaneously, meaning that no special feedback is necessary except to display the result. <br />1.0 second is about the limit for the user&apos;s flow of thought to stay uninterrupted, even though the user will notice the delay. Normally, no special feedback is necessary during delays of more than 0.1 but less than 1.0 second, but the user does lose the feeling of operating directly on the data. <br />10 seconds is about the limit for keeping the user&apos;s attention focused on the dialogue. For longer delays, users will want to perform other tasks while waiting for the computer to finish, so they should be given feedback indicating when the computer expects to be done. Feedback during the delay is especially important if the response time is likely to be highly variable, since users will then not know what to expect.&quot; <br />
  28. 28. New Alternatives to HTTP Layer Testing<br />Browser-based load testing.<br />Historically, not very scalable.<br />Enter the cloud<br />The cloud makes browser based load testing scalable. <br />LoadStorm<br />
  29. 29. Benefits<br />Script maintainability<br />Changes to the websites AJAX code will flow through as JavaScript is executed at test-time.<br />Proper handling of: <br />Changes <br />to Polling intervals<br />New AJAX requests<br />Deleted AJAX requests<br />Auto-complete fields <br />
  30. 30. When to use browser-based testing<br />When you have an Ajax heavy or dynamic site<br />When you have flash or difficult-to-reproduce interactions.<br />When your man-hours are precious<br />Easier maintenance and development make it worthwhile<br />Generally affordable for small tests, run frequently<br />When you can! <br />
  31. 31. When not to used browser-based testing<br />When you need to test 200,000 users.<br />Cost prohibitive<br />When you cannot expose your site to external traffic (intranet)<br />There are commercial load testing tools that let you run real browsers locally. <br />For smaller tests, this can work.<br />
  1. A particular slide catching your eye?

    Clipping is a handy way to collect important slides you want to go back to later.

×