0
Bring Your Smoke Tests  to Life Michael Peters (mpeters@plusthree.com) http://sourceforge.net/projects/smolder
Problems <ul><li>Large and long running test suites? </li></ul><ul><li>Code changes without test runs? </li></ul><ul><li>M...
Origins <ul><li>5460 tests </li></ul><ul><li>28 minutes </li></ul><ul><li>5 different dev platforms </li></ul>PlusThree an...
Origins <ul><li>Automated build and test server </li></ul><ul><ul><ul><li>emailed results every morning </li></ul></ul></u...
Origins PlusThree and Arcos <ul><li>Several variations on test runs </li></ul><ul><li>15,000+ tests </li></ul><ul><li>40+ ...
Origins <ul><li>Lot's of ideas from Pugs smoke server </li></ul><ul><li>Developers customize </li></ul><ul><ul><ul><li>Ema...
Origins Built With <ul><li>Apache/mod_perl </li></ul><ul><li>CGI::Application and family </li></ul><ul><li>Template Toolki...
TAP <ul><li>Test Anything Protocol </li></ul><ul><ul><ul><ul><li>http://search.cpan.org/perldoc?TAP </li></ul></ul></ul></...
TAP 1..5 ok 1 – foo is fine ok 2 – testing bar value not ok 3 – The object isa Frobnicate # this is some useful # comment ...
TAP use Test::More (tests => 5) ok($foo, 'foo is fine'); is($bar, 'right', 'testing bar value'); isa_ok($bar, 'Frobnicate'...
TAP <ul><li>Started in Perl-land </li></ul><ul><ul><ul><li>language-agnostic </li></ul></ul></ul><ul><ul><ul><ul><li>C </l...
TAP <ul><li>A single archive file (currently Tar) </li></ul><ul><li>One file for each TAP  stream </li></ul><ul><li>A YAML...
Using <ul><li>How often? </li></ul><ul><ul><li>Every checkin? </li></ul></ul><ul><ul><li>Periodically (every hour) ? </li>...
Using <ul><li>Every Checkin </li></ul><ul><ul><ul><li>SCM and build/test on same server </li></ul></ul></ul><ul><ul><ul><l...
Using <ul><li>Every Hour </li></ul><ul><ul><ul><li>cron job </li></ul></ul></ul><ul><ul><ul><ul><li>setup environment/serv...
Future <ul><li>RSS/Atom feeds in addition to email </li></ul><ul><li>Better integration with  TestRunner::Multi </li></ul>...
Future <ul><li>Code coverage reports and trends </li></ul><ul><li>??? </li></ul>
Upcoming SlideShare
Loading in...5
×

Smolder Introduction

4,129

Published on

A quick intro to Smolder.

Published in: Technology
1 Comment
4 Likes
Statistics
Notes
  • I can't wait to see more presentations from you.
    http://www.teethgrindinginsleep.net/
    http://www.teethgrindinginsleep.net/how-to-stop-teeth-grinding-at-night/
    http://www.teethgrindinginsleep.net/get-a-mouth-guard-to-stop-teeth-grinding-at-night/
       Reply 
    Are you sure you want to  Yes  No
    Your message goes here
No Downloads
Views
Total Views
4,129
On Slideshare
0
From Embeds
0
Number of Embeds
0
Actions
Shares
0
Downloads
57
Comments
1
Likes
4
Embeds 0
No embeds

No notes for slide

Transcript of "Smolder Introduction"

  1. 1. Bring Your Smoke Tests to Life Michael Peters (mpeters@plusthree.com) http://sourceforge.net/projects/smolder
  2. 2. Problems <ul><li>Large and long running test suites? </li></ul><ul><li>Code changes without test runs? </li></ul><ul><li>Multiple languages </li></ul><ul><ul><ul><li>How do you bring them all together? </li></ul></ul></ul><ul><li>Project managers need to see pretty graphs? </li></ul>
  3. 3. Origins <ul><li>5460 tests </li></ul><ul><li>28 minutes </li></ul><ul><li>5 different dev platforms </li></ul>PlusThree and PlunkIT
  4. 4. Origins <ul><li>Automated build and test server </li></ul><ul><ul><ul><li>emailed results every morning </li></ul></ul></ul><ul><ul><ul><ul><li>always </li></ul></ul></ul></ul><ul><ul><ul><ul><li>huge (510k of HTML) </li></ul></ul></ul></ul><ul><ul><ul><ul><li>developers had no control </li></ul></ul></ul></ul>PlusThree and PlunkIT
  5. 5. Origins PlusThree and Arcos <ul><li>Several variations on test runs </li></ul><ul><li>15,000+ tests </li></ul><ul><li>40+ minutes </li></ul><ul><li>It was annoying and wasteful </li></ul>
  6. 6. Origins <ul><li>Lot's of ideas from Pugs smoke server </li></ul><ul><li>Developers customize </li></ul><ul><ul><ul><li>Email frequency </li></ul></ul></ul><ul><ul><ul><li>Email format </li></ul></ul></ul><ul><ul><ul><li>Raw data </li></ul></ul></ul><ul><li>Tests can be commented, categorized and invalidated </li></ul><ul><li>Trends can be viewed </li></ul><ul><li>Can host multiple projects </li></ul><ul><ul><ul><li>public and private </li></ul></ul></ul>
  7. 7. Origins Built With <ul><li>Apache/mod_perl </li></ul><ul><li>CGI::Application and family </li></ul><ul><li>Template Toolkit </li></ul><ul><li>SQLite (or MySQL if you prefer) </li></ul><ul><li>Prototype.js (lots 'o AJAX) </li></ul><ul><ul><ul><li>Full reports are lazy loaded </li></ul></ul></ul>
  8. 8. TAP <ul><li>Test Anything Protocol </li></ul><ul><ul><ul><ul><li>http://search.cpan.org/perldoc?TAP </li></ul></ul></ul></ul><ul><li>It's what test scripts output </li></ul><ul><ul><ul><ul><li>Test::Builder, Test::More, etc </li></ul></ul></ul></ul>
  9. 9. TAP 1..5 ok 1 – foo is fine ok 2 – testing bar value not ok 3 – The object isa Frobnicate # this is some useful # comment thing not ok 4 # SKIP Foo::Bar is not installed not ok 5 # TODO future feature
  10. 10. TAP use Test::More (tests => 5) ok($foo, 'foo is fine'); is($bar, 'right', 'testing bar value'); isa_ok($bar, 'Frobnicate'); diag(“this is some useful comment thing”); SKIP: { skip('Foo::Bar is not installed' 1); ok($foo->bar, 'has a bar'); } TODO: { local $TODO = 'future feature'; ok($foo->feature, 'some feature'); }
  11. 11. TAP <ul><li>Started in Perl-land </li></ul><ul><ul><ul><li>language-agnostic </li></ul></ul></ul><ul><ul><ul><ul><li>C </li></ul></ul></ul></ul><ul><ul><ul><ul><li>PHP </li></ul></ul></ul></ul><ul><ul><ul><ul><li>Python </li></ul></ul></ul></ul><ul><ul><ul><ul><li>Java </li></ul></ul></ul></ul><ul><ul><ul><ul><li>Javascript </li></ul></ul></ul></ul><ul><ul><ul><ul><li>Parrot </li></ul></ul></ul></ul>
  12. 12. TAP <ul><li>A single archive file (currently Tar) </li></ul><ul><li>One file for each TAP stream </li></ul><ul><li>A YAML file containing meta information about the test run </li></ul><ul><ul><ul><li>Order </li></ul></ul></ul><ul><ul><ul><li>Start and stop times </li></ul></ul></ul><ul><ul><ul><li>Future expansion </li></ul></ul></ul>TAP Archives
  13. 13. Using <ul><li>How often? </li></ul><ul><ul><li>Every checkin? </li></ul></ul><ul><ul><li>Periodically (every hour) ? </li></ul></ul>Automated Tests
  14. 14. Using <ul><li>Every Checkin </li></ul><ul><ul><ul><li>SCM and build/test on same server </li></ul></ul></ul><ul><ul><ul><li>checkin-hook script </li></ul></ul></ul><ul><ul><ul><ul><li>setup environment/services </li></ul></ul></ul></ul><ul><ul><ul><ul><li>use TestRunner::Multi (module by Dave Rolsky) </li></ul></ul></ul></ul>Automated Tests
  15. 15. Using <ul><li>Every Hour </li></ul><ul><ul><ul><li>cron job </li></ul></ul></ul><ul><ul><ul><ul><li>setup environment/services </li></ul></ul></ul></ul><ul><ul><ul><ul><li>Use TestRunner::Multi </li></ul></ul></ul></ul>Automated Tests
  16. 16. Future <ul><li>RSS/Atom feeds in addition to email </li></ul><ul><li>Better integration with TestRunner::Multi </li></ul>Upcoming Features
  17. 17. Future <ul><li>Code coverage reports and trends </li></ul><ul><li>??? </li></ul>
  1. A particular slide catching your eye?

    Clipping is a handy way to collect important slides you want to go back to later.

×