Smolder And Buildbot

What is Smolder? ● Will it automate my build/test cycle? – No. ● Will it run my test suite? – No. ● Will it checkout my code from svn, run my build and then run my test suite? – Hello! Smolder takes the information about a test run and makes it pretty. Michael Peters, Plus Three LP (http://plusthree.com)

History of Smolder ● Plus Three, LP ● Long running test suites for multiple projects – 5,460 tests in 28 minutes (510K HTML report) – 18,000+ tests in 50 minutes ● Stole lots of ideas from the Pugs smoke server. ● Added lot – Comments, tags, invalidation – Pretty graphs – Multiple projects (public and private) – Atom data feeds Michael Peters, Plus Three LP (http://plusthree.com)

TAP ● Test Anything Protocol ● Created by Test::Building, Test::More, etc ● Consumed by Test::Harness, prove, etc 1..5 ok 1 – foo is fine ok 2 – testing bar value not ok 3 – The object isa Frobnicate # this is some useful # comment thing not ok 4 # SKIP Foo::Bar is not instal led not ok 5 # TODO future feature Michael Peters, Plus Three LP (http://plusthree.com)

TAP ● How do you create it? use Test::More (tests => 5) ok($foo, 'foo is fine'); is($bar, 'right', 'testing bar value'); isa_ok($bar, 'Frobnicate'); diag(“this is some usefulncomment thing”); SKIP: { skip('Foo::Bar is not installed' 1); ok($foo>bar, 'has a bar'); } TODO: { local $TODO = 'future feature'; ok($foo>feature, 'some feature'); } Michael Peters, Plus Three LP (http://plusthree.com)

TAP ● Started in Perl-Land ● We have since created colonies – C – PHP – Python – Java – JavaScript – Parrot – Postgres – .... more Michael Peters, Plus Three LP (http://plusthree.com)

TAP Archives ● A single file (.tar or .tar.gz) ● Multiple recorded TAP streams (files) ● Some meta information (META.yml) – Stream (test) order – Start/Stop times – Future expansion (server info, log files, etc) ● Easy to create with TAP::Harness::Archive – prove -a Michael Peters, Plus Three LP (http://plusthree.com)

Continuous Integration Cycle ● Automated builds – Simple bash script #!/bin/bash if svn update | grep 'Updated to' then make build && make test else echo quot;No updatesquot; fi ● Get fancier – SmokeRunner::Multi – Custom harness written with Test::Harness – BuildBot Michael Peters, Plus Three LP (http://plusthree.com)

Buildbot ● Most projects don't have multi-platform, multi- architecture needs – Anything written in pure Perl – Most $work applications ● Some projects do – Languages (Perl, Python) – VMs (Parrot) – System Tools (SCM, etc) Michael Peters, Plus Three LP (http://plusthree.com)

What is Buildbot? ● Will it automate my build/test cycle? – Yes. ● Will it run my test suite? – Yes. ● Will it checkout my code from svn, run my build and then run my test suite? – Absolutely! Michael Peters, Plus Three LP (http://plusthree.com)

Other Buildbot Spiffyness ● Master configuration, multiple slaves do the work – Lots of architectures and platforms ● Arbitrary build commands – Not a language specific system – Doesn't care if you use make, Build.PL, ant, maven ● Portable – It's Python and works in lots of places despite all the whitespace ● Very configurable Michael Peters, Plus Three LP (http://plusthree.com)

So Why Do I Need Smolder? ● Buildbot gives just a single pass/fail for each run ● All other details are just dumped to the screen ● No good way to organize your test runs ● .... Michael Peters, Plus Three LP (http://plusthree.com)

So Why Do I Need Smolder? ● It's UGLY! Michael Peters, Plus Three LP (http://plusthree.com)

So Why Do I Need Smolder? ● Oooh-la-la! Michael Peters, Plus Three LP (http://plusthree.com)

So Why Do I Need Smolder? Buildbot => Smolder Michael Peters, Plus Three LP (http://plusthree.com)

So Why Do I Need Smolder? Buildbot => Smolder => Michael Peters, Plus Three LP (http://plusthree.com)

So Why Do I Need Smolder? ● Information is more organized – Access to the raw TAP Archives – Search interface and Graphs ● Let's you answer questions like – When was the last time our test suite passed on platform X? – Who broke the test suite? – How many tests have we added for this release? – How did checkin X affect the speed of the test suite? Michael Peters, Plus Three LP (http://plusthree.com)

Buildbot Talking to Smolder ● Remember Buildbot allows arbitrary build commands ● So your project just needs a command that can run the suite, create a TAP archive and send it to a Smolder server – prove + smolder_smoke_signal – make smoke_tests ● Buildbot doesn't know that it's being cheated on! Michael Peters, Plus Three LP (http://plusthree.com)

Scary Live Demos! ● Parrot ● Smolder Michael Peters, Plus Three LP (http://plusthree.com)

Smolder's Future ● Test Coverage ● Tracking individual test assertions ● Better visual display of diagnostics (new TAP extensions) ● More and better ways to search old reports Michael Peters, Plus Three LP (http://plusthree.com)

Smolder And Buildbot

by Michael Peters on Oct 15, 2008

Accessibility

Categories

Upload Details

Usage Rights

1 Embed 9

Statistics

Smolder And Buildbot — Presentation Transcript