Smolder Introduction

Bring Your Smoke Tests to Life Michael Peters (mpeters@plusthree.com) http://sourceforge.net/projects/smolder

Problems
Large and long running test suites?
Code changes without test runs?
Multiple languages
How do you bring them all together?
Project managers need to see pretty graphs?

Origins
5460 tests
28 minutes
5 different dev platforms
PlusThree and PlunkIT

Origins
Automated build and test server
emailed results every morning
always
huge (510k of HTML)
developers had no control
PlusThree and PlunkIT

Origins PlusThree and Arcos
Several variations on test runs
15,000+ tests
40+ minutes
It was annoying and wasteful

Origins
Lot's of ideas from Pugs smoke server
Developers customize
Email frequency
Email format
Raw data
Tests can be commented, categorized and invalidated
Trends can be viewed
Can host multiple projects
public and private

Origins Built With
Apache/mod_perl
CGI::Application and family
Template Toolkit
SQLite (or MySQL if you prefer)
Prototype.js (lots 'o AJAX)
Full reports are lazy loaded

TAP
Test Anything Protocol
http://search.cpan.org/perldoc?TAP
It's what test scripts output
Test::Builder, Test::More, etc

TAP 1..5 ok 1 – foo is fine ok 2 – testing bar value not ok 3 – The object isa Frobnicate # this is some useful # comment thing not ok 4 # SKIP Foo::Bar is not installed not ok 5 # TODO future feature

TAP use Test::More (tests => 5) ok($foo, 'foo is fine'); is($bar, 'right', 'testing bar value'); isa_ok($bar, 'Frobnicate'); diag(“this is some useful comment thing”); SKIP: { skip('Foo::Bar is not installed' 1); ok($foo->bar, 'has a bar'); } TODO: { local $TODO = 'future feature'; ok($foo->feature, 'some feature'); }

TAP
Started in Perl-land
language-agnostic
C
PHP
Python
Java
Javascript
Parrot

TAP
A single archive file (currently Tar)
One file for each TAP stream
A YAML file containing meta information about the test run
Order
Start and stop times
Future expansion
TAP Archives

Using
How often?
Every checkin?
Periodically (every hour) ?
Automated Tests

Using
Every Checkin
SCM and build/test on same server
checkin-hook script
setup environment/services
use TestRunner::Multi (module by Dave Rolsky)
Automated Tests

Using
Every Hour
cron job
setup environment/services
Use TestRunner::Multi
Automated Tests

Future
RSS/Atom feeds in addition to email
Better integration with TestRunner::Multi
Upcoming Features

Future
Code coverage reports and trends
???

Smolder Introduction

by Michael Peters on Aug 30, 2007

Accessibility

Categories

Tags

Upload Details

Usage Rights

1 Embed 23

Statistics

Smolder Introduction — Presentation Transcript