Testing Toolbox

Updating Your Testing Toolbox Michael Peters PlusThree, LP http://www.plusthree.com/ OSCON 2007

Difficulties
Hard to get time scheduled
Hard to get non-programmers involved
Hard to see progress

Testing
During Development
Performance
Regression (Smoke)

During Development
Firebug
Firefox Plugin
JS Console and Debugger
HTML/CSS/DOM Inspector
XHR inspector

During Development
Firebug
Firefox Only
Firebug Lite ( http://www.getfirebug.com/lite.html )
Safari 3 will have Web Inspector
IE has VWD, but it's not even close

During Development
LiveHTTPHeaders
Another Firefox plugin
All HTTP headers for All requests
Cookies, X-JSON, Cache, etc

During Development
View Formatted Source
View Source is static

During Development
View Formatted Source
View Source is static
View Formatted Source is ALIVE!

During Development
PageDiff
Capture the state of the current page
View the difference after some actions

Performance
Load Time Analyzer
Why does this page take so long to load?
Does it really matter how big this JS file is?

Performance Load Time Analyzer
vs Built-in Firebug Net Inspector
LTA is a printable
LTA gives more information about JS execution time after it's loaded
Pingdom – a similar online tool

Performance Yslow vs GTA
More than just time
Gives tips and hints
Summarized by topic

Performance benchmark.js
http://gfx.neohub.com/benchmark/
Similar to Benchmark in Perl
Compare similar techniques to find the fastest
See performance differences between browsers
Compare different JS libs that do similar things

Performance benchmark.js
vs Firebug's built-in JS profiler
Benchmark is for smaller examples
Profiler for real code/pages
benchmark.js works in multiple browsers

Regression (Smoke) Selenium RC
http://www.openqa.org/selenium-rc/
Selenium Server
Java program that can start/stop/control a browser
IE 6, 7
FF 1.5, 2.0
Opera 8.5, 9.0
Safari 1.3, 2.0+
Selenium Core
Javascript libraries to control a browser's behavior and make requests

Regression (Smoke) Selenium RC
http://www.openqa.org/selenium-rc/
WWW::Selenium
Control the browser
Test::WWW::Selenium
subclasses WWW::Selenium to add testing methods

Regression (Smoke) http://www.openqa.org/selenium-ide/ Selenium IDE
Record actions in the browser (Firefox)
Add assertions
Save test case (a simple HTML format)
Run test cases

Regression (Smoke) Selenium RC + IDE
Anyone can record test cases
Save those cases in a shared location
Anyone can run those cases
Can be done so that cases are run on different OSes/browsers
Very powerful if you allow non-programmers to reset test data

Regression (Smoke) Smolder
At PlusThree we have some large projects with large test suites
15,000+ tests (200+ .t files)
50+ minutes
Doesn't let you use that box for anything else
Run automated tests on a dedicated machine periodically if there have been changes

Tests can be commented, categoried and invalidated
Trends can be viewed with graphs
Individual test runs have interactive reports

Can host multiple projects
public and private
Developers can customize email they receive
frequency and format
Public projects can also provide data feeds
Atom

Regression (Smoke) TAP
Test Anything Protocol
Simple, line oriented
Almost 20 years old
Language Agnostic
C, C++, Forth, Javascript, PHP, Perl, PostgreSQL,Python, Ruby, PIR

Regression (Smoke) TAP
In Perl
Test::Builder, Test::More, etc
Test::Harness
TAP::Parser

Regression (Smoke) TAP + Smolder
TAP Archive
An archive of .tap files: one for each test run
An optional YAML file containing meta information about the test run
Start and end times
What order the files were run in
TestRunner::Multi
runs tests from source control

All Together Now
Developers write tests
Perl – Test::More, Test::WWW::Mechanize, et c
Automated test harness runs them
QA/Acceptance Testers, Product Managers record tests
Browser – Selenium IDE
Automated test harness runs them
TAP is produced
Smolder creates pretty reports, graphs and email
Everyone feels involved
= Shiny Happy Users :)

http://www.slideshare.net	17
http://www.techgig.com	9
http://10.150.200.57	1
http://115.112.206.131	1

Testing Toolbox

by Michael Peters on Aug 30, 2007

Accessibility

Categories

Upload Details

Usage Rights

4 Embeds 28

Statistics

Testing Toolbox Presentation Transcript