Testing Toolbox

  • 23,089 views
Uploaded on

Presentation at OSCON 2007 by Michael Peters about various web testing tools.

Presentation at OSCON 2007 by Michael Peters about various web testing tools.

More in: Technology
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Be the first to comment
No Downloads

Views

Total Views
23,089
On Slideshare
0
From Embeds
0
Number of Embeds
1

Actions

Shares
Downloads
155
Comments
0
Likes
2

Embeds 0

No embeds

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
    No notes for slide

Transcript

  • 1. Updating Your Testing Toolbox Michael Peters PlusThree, LP http://www.plusthree.com/ OSCON 2007
  • 2. Difficulties
    • Hard to get time scheduled
    • Hard to get non-programmers involved
    • Hard to see progress
  • 3. Testing
    • During Development
    • Performance
    • Regression (Smoke)
  • 4. ...
  • 5. During Development
    • Firebug
    • Firefox Plugin
    • JS Console and Debugger
    • HTML/CSS/DOM Inspector
    • XHR inspector
  • 6. During Development
    • Firebug
    • Firefox Only
    • Firebug Lite ( http://www.getfirebug.com/lite.html )
    • Safari 3 will have Web Inspector
    • IE has VWD, but it's not even close
  • 7. During Development
    • LiveHTTPHeaders
    • Another Firefox plugin
    • All HTTP headers for All requests
    • Cookies, X-JSON, Cache, etc
  • 8. During Development
    • View Formatted Source
    • View Source is static
  • 9. During Development
    • View Formatted Source
    • View Source is static
    • View Formatted Source is ALIVE!
  • 10. During Development
    • PageDiff
    • Capture the state of the current page
    • View the difference after some actions
  • 11. ...
  • 12. Performance
    • Load Time Analyzer
    • Why does this page take so long to load?
    • Does it really matter how big this JS file is?
  • 13. Performance Load Time Analyzer
    • vs Built-in Firebug Net Inspector
    • LTA is a printable
    • LTA gives more information about JS execution time after it's loaded
    Pingdom – a similar online tool
  • 14. Performance Yslow vs GTA
    • More than just time
    • Gives tips and hints
    • Summarized by topic
  • 15. Performance benchmark.js
    • http://gfx.neohub.com/benchmark/
    • Similar to Benchmark in Perl
    • Compare similar techniques to find the fastest
    • See performance differences between browsers
    • Compare different JS libs that do similar things
  • 16. Performance benchmark.js
    • vs Firebug's built-in JS profiler
    • Benchmark is for smaller examples
    • Profiler for real code/pages
    • benchmark.js works in multiple browsers
  • 17. ...
  • 18. Regression (Smoke) Selenium RC
    • http://www.openqa.org/selenium-rc/
    • Selenium Server
      • Java program that can start/stop/control a browser
          • IE 6, 7
          • FF 1.5, 2.0
          • Opera 8.5, 9.0
          • Safari 1.3, 2.0+
    • Selenium Core
      • Javascript libraries to control a browser's behavior and make requests
  • 19. Regression (Smoke) Selenium RC
    • http://www.openqa.org/selenium-rc/
    • WWW::Selenium
        • Control the browser
    • Test::WWW::Selenium
        • subclasses WWW::Selenium to add testing methods
  • 20. Regression (Smoke) http://www.openqa.org/selenium-ide/ Selenium IDE
    • Record actions in the browser (Firefox)
    • Add assertions
    • Save test case (a simple HTML format)
    • Run test cases
  • 21. Regression (Smoke) Selenium RC + IDE
    • Anyone can record test cases
    • Save those cases in a shared location
    • Anyone can run those cases
      • Can be done so that cases are run on different OSes/browsers
    • Very powerful if you allow non-programmers to reset test data
  • 22. Regression (Smoke) Smolder
    • At PlusThree we have some large projects with large test suites
    • 15,000+ tests (200+ .t files)
    • 50+ minutes
    • Doesn't let you use that box for anything else
    • Run automated tests on a dedicated machine periodically if there have been changes
  • 23. Regression (Smoke) Smolder
    • Tests can be commented, categoried and invalidated
    • Trends can be viewed with graphs
    • Individual test runs have interactive reports
  • 24. Regression (Smoke) Smolder
    • Can host multiple projects
      • public and private
    • Developers can customize email they receive
      • frequency and format
    • Public projects can also provide data feeds
      • Atom
  • 25. Regression (Smoke) TAP
    • Test Anything Protocol
    • Simple, line oriented
    • Almost 20 years old
    • Language Agnostic
    C, C++, Forth, Javascript, PHP, Perl, PostgreSQL,Python, Ruby, PIR
  • 26. Regression (Smoke) TAP
    • In Perl
    • Test::Builder, Test::More, etc
    • Test::Harness
    • TAP::Parser
  • 27. Regression (Smoke) TAP + Smolder
    • TAP Archive
    • An archive of .tap files: one for each test run
    • An optional YAML file containing meta information about the test run
      • Start and end times
      • What order the files were run in
    • TestRunner::Multi
      • runs tests from source control
  • 28. All Together Now
    • Developers write tests
        • Perl – Test::More, Test::WWW::Mechanize, et c
    • Automated test harness runs them
    • QA/Acceptance Testers, Product Managers record tests
        • Browser – Selenium IDE
    • Automated test harness runs them
    • TAP is produced
    • Smolder creates pretty reports, graphs and email
    • Everyone feels involved
    • = Shiny Happy Users :)
  • 29. ...